Welcome to WebmasterWorld Guest from

Forum Moderators: open

Message Too Old, No Replies

fast webcrawler crawls only my startpage

Is it because the address of my website is a subdomain?



12:41 pm on Mar 15, 2003 (gmt 0)

10+ Year Member

I noticed that googlebot and inktomi are going for a deepcrawl on my website right now. But what about the fast crawler, it crawled only my startpage "/" twice so far. Is that crawler not going for a deepcrawl because my site's address is a subdomain (of a ISP)?
The first time fast crawled my startpage was February 27th. (I launched my site in January)

Anybody any ideas? Should I just be more patient?


2:35 pm on Mar 15, 2003 (gmt 0)

WebmasterWorld Senior Member heini is a WebmasterWorld Top Contributor of All Time 10+ Year Member

Elord, I'm afraid you need to be more patient. Fast doesn't crawl regularly and fast enough, and that's about the only major gripe I have with them.
To really put it off with Google they need to take a major step forward in this regard.
Technically it shouldn't be a problem for them. A little more infrastructure, and man power, very likely would do the trick.
Taking only the index page for a while, and then in several steps the rest of the site is a pattern I've seen for a while now with Fast.


2:48 pm on Mar 15, 2003 (gmt 0)

10+ Year Member

I see, thank you, heini.
Right after my post I looked at statistics of another website, there fast only crawled one page, too.
Well, I guess I just need to give fast a little more time.


5:47 pm on Mar 15, 2003 (gmt 0)

10+ Year Member

elord, I agree with heini. Fast is not fast, but they eventually get everything.

Featured Threads

Hot Threads This Week

Hot Threads This Month