Welcome to WebmasterWorld Guest from

Forum Moderators: open

Message Too Old, No Replies

fast webcrawler crawls only my startpage

Is it because the address of my website is a subdomain?

12:41 pm on Mar 15, 2003 (gmt 0)

New User

10+ Year Member

joined:Jan 15, 2003
votes: 0

I noticed that googlebot and inktomi are going for a deepcrawl on my website right now. But what about the fast crawler, it crawled only my startpage "/" twice so far. Is that crawler not going for a deepcrawl because my site's address is a subdomain (of a ISP)?
The first time fast crawled my startpage was February 27th. (I launched my site in January)

Anybody any ideas? Should I just be more patient?

2:35 pm on Mar 15, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member heini is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Jan 31, 2001
votes: 0

Elord, I'm afraid you need to be more patient. Fast doesn't crawl regularly and fast enough, and that's about the only major gripe I have with them.
To really put it off with Google they need to take a major step forward in this regard.
Technically it shouldn't be a problem for them. A little more infrastructure, and man power, very likely would do the trick.
Taking only the index page for a while, and then in several steps the rest of the site is a pattern I've seen for a while now with Fast.
2:48 pm on Mar 15, 2003 (gmt 0)

New User

10+ Year Member

joined:Jan 15, 2003
votes: 0

I see, thank you, heini.
Right after my post I looked at statistics of another website, there fast only crawled one page, too.
Well, I guess I just need to give fast a little more time.
5:47 pm on Mar 15, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Jan 15, 2003
votes: 0

elord, I agree with heini. Fast is not fast, but they eventually get everything.

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members