Jos,
certainly doable in vfp.
Think more about scalabilty, expected amout of change,
bandwidth, expected download volume, user know how to decide on your approach.

IE-based spidering is very easy if you have to decide on certain paths/patterns
or often changing side, since it is nearly "visual" programming.
IE parsing is nice, but is slower than direct string work
(but most of the time the bottleneck is receiving the data)

pure brute force attacks on a (rather) small line should be done via
WinInet(wrapper) - see westwind, afp and some other libraries.
There used to be also MSDN article, and there is UrlDownloadToFile()...

One thing: since vfp is single threaded, a threaded approach,
which might be sensible in some use cases, is rather difficult.
This would be reason for me to jump to .net, even if really like the
fox - they have some nice classes there in fine granularity.

One other thing: Hope you are using a late vfp version:
I started doing this in vfp5 and IE4:
hell in large doses, so I switched over to vb and saving
via vfp.com. No inheritance, but I was able to work with a few
simple strategy patterns. Later on vfp6 I revisited the project
(much more doable without vb <g>) but since it was working,
I left it in vb. Would have been a near toss up for a new project.

With vfp8 I won't think about vb (but at least consider .net)

my 0.02 EUR

thomas