Welcome to WebmasterWorld Guest from

Forum Moderators: open

fast webcrawler crawls only my startpage

Is it because the address of my website is a subdomain?

12:41 pm on Mar 15, 2003 (gmt 0)

10+ Year Member

I noticed that googlebot and inktomi are going for a deepcrawl on my website right now. But what about the fast crawler, it crawled only my startpage "/" twice so far. Is that crawler not going for a deepcrawl because my site's address is a subdomain (of a ISP)?
The first time fast crawled my startpage was February 27th. (I launched my site in January)

Anybody any ideas? Should I just be more patient?

2:35 pm on Mar 15, 2003 (gmt 0)

WebmasterWorld Senior Member heini is a WebmasterWorld Top Contributor of All Time 10+ Year Member

Elord, I'm afraid you need to be more patient. Fast doesn't crawl regularly and fast enough, and that's about the only major gripe I have with them.
To really put it off with Google they need to take a major step forward in this regard.
Technically it shouldn't be a problem for them. A little more infrastructure, and man power, very likely would do the trick.
Taking only the index page for a while, and then in several steps the rest of the site is a pattern I've seen for a while now with Fast.
2:48 pm on Mar 15, 2003 (gmt 0)

10+ Year Member

I see, thank you, heini.
Right after my post I looked at statistics of another website, there fast only crawled one page, too.
Well, I guess I just need to give fast a little more time.
5:47 pm on Mar 15, 2003 (gmt 0)

10+ Year Member

elord, I agree with heini. Fast is not fast, but they eventually get everything.

Featured Threads

My Threads

Hot Threads This Week

Hot Threads This Month

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved