Welcome to WebmasterWorld Guest from

Forum Moderators: open

Message Too Old, No Replies

spidering speed

How fast is fast...

12:48 pm on Dec 29, 2002 (gmt 0)

Full Member

10+ Year Member

joined:Oct 16, 2002
votes: 0

Seems that I have been visiting by the Fast Webcrawler, it only picked up my robots.txt file.

When is it likely that it will crawl through the html-pages?

2:06 pm on Dec 31, 2002 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
votes: 28

It should happen within 7 days to two weeks. If it is a new site, it may be up to one month before it is picked up entirely.