homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Visit PubCon.com
Home / Forums Index / Yahoo / Deprecated - Altavista, Alltheweb.com
Forum Library, Charter, Moderator: open

Deprecated - Altavista, Alltheweb.com Forum

fast webcrawler crawls only my startpage
Is it because the address of my website is a subdomain?

 12:41 pm on Mar 15, 2003 (gmt 0)

I noticed that googlebot and inktomi are going for a deepcrawl on my website right now. But what about the fast crawler, it crawled only my startpage "/" twice so far. Is that crawler not going for a deepcrawl because my site's address is a subdomain (of a ISP)?
The first time fast crawled my startpage was February 27th. (I launched my site in January)

Anybody any ideas? Should I just be more patient?



 2:35 pm on Mar 15, 2003 (gmt 0)

Elord, I'm afraid you need to be more patient. Fast doesn't crawl regularly and fast enough, and that's about the only major gripe I have with them.
To really put it off with Google they need to take a major step forward in this regard.
Technically it shouldn't be a problem for them. A little more infrastructure, and man power, very likely would do the trick.
Taking only the index page for a while, and then in several steps the rest of the site is a pattern I've seen for a while now with Fast.


 2:48 pm on Mar 15, 2003 (gmt 0)

I see, thank you, heini.
Right after my post I looked at statistics of another website, there fast only crawled one page, too.
Well, I guess I just need to give fast a little more time.


 5:47 pm on Mar 15, 2003 (gmt 0)

elord, I agree with heini. Fast is not fast, but they eventually get everything.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Yahoo / Deprecated - Altavista, Alltheweb.com
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved