Forum Moderators: open

Message Too Old, No Replies

BaiduSpider Deep Crawl?

         

c3p0

9:08 pm on Sep 18, 2004 (gmt 0)

10+ Year Member



I am using AWstats to analyze logs. I am not an expert on AWstats, but I see in the report that BaiDuSpider visited my website. It only shows 1 "Hit" from that spider though. I am assuming that this means the spider visited only 1 page, but did not follow any of the links. Is this normal or is there likely something wrong with my page? My pages does validate.

Also, beside BaiDuSpider, is it normal for other search engine bots to only generate 1 hit? I see that Askjeeves generated a "1+1" (meaning 1 hit plus 1 successful robots.txt request i think). However, the Inktomi slurp generated 26 hits.

Thanks.

bill

12:33 am on Sep 19, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



It seems like you only had one hit from that spider. It's very hard to say why you only had one hit. Some spiders will only make a few hits on an initial visit and then come back for a deeper crawl. Maybe it was following a link to your site while spidering another site. It could be something else. I'd check your raw logs and see what page was hit, then I'd wait to see if the problem persists. I don't know what the exact schedule of the Baidu spider is, so you may have to be patient for a while.