| fast webcrawler crawls only my startpage
Is it because the address of my website is a subdomain? |
elord
msg:224346 | 12:41 pm on Mar 15, 2003 (gmt 0) |
Hello,
I noticed that googlebot and inktomi are going for a deepcrawl on my website right now. But what about the fast crawler, it crawled only my startpage "/" twice so far. Is that crawler not going for a deepcrawl because my site's address is a subdomain (of a ISP)?
The first time fast crawled my startpage was February 27th. (I launched my site in January) Anybody any ideas? Should I just be more patient?
Thanks.
|
heini
msg:224347 | 2:35 pm on Mar 15, 2003 (gmt 0) |
Elord, I'm afraid you need to be more patient. Fast doesn't crawl regularly and fast enough, and that's about the only major gripe I have with them.
To really put it off with Google they need to take a major step forward in this regard.
Technically it shouldn't be a problem for them. A little more infrastructure, and man power, very likely would do the trick.
Taking only the index page for a while, and then in several steps the rest of the site is a pattern I've seen for a while now with Fast.
|
elord
msg:224348 | 2:48 pm on Mar 15, 2003 (gmt 0) |
I see, thank you, heini.
Right after my post I looked at statistics of another website, there fast only crawled one page, too.
Well, I guess I just need to give fast a little more time.
|
rogue
msg:224349 | 5:47 pm on Mar 15, 2003 (gmt 0) |
elord, I agree with heini. Fast is not fast, but they eventually get everything.
|
|
|
