Strange accesslog finding

[epmaniac]

hi

i noticed a very strange behaviour with my accesslog today.

99% of the pages which are crawled by all bots (google,yahoo,bing) are all pages which have words in link with 3 letters or less

for example:

http://www.example.com/ men-gifts/
http://www.example.com/ ipod-mp4/
www.example.com/ mlb-jerseys/
http://www.example.com/ -mlb-logo/
http://www.example.com/ -nike-af/


is it a natural behaviour or could it be due to some error? please guide me, i think something is terribly wrong

[epmaniac]

one more thing

for words with length 3 0r less than 3, i search with LIKE in database,... for greater than 3 i use full text search.... i dont know how this could be affecting the crawling of bots, but maybe it is

[cls_wired]

There is little probability, that all bots can start to crawl 3-letters words in one day. Problem is on the site's engine side.

[epmaniac]

this problem i have seen in all my previous accesslogs too, my site went from being one of top 40000 site to being obscure

[deadsea]

Googlebot usually recrawls pages with a frequency proportional to the pagerank of the page. However, I have seen googlbot do a much deeper crawl. In this mode it visits urls that it may not have visited in years, or may not have visited at all in the past. It kind of looks like they found a whole box of urls in the basement and open it up like a kid a christmas. Interestingly, googlebot seems to crawl urls by url length, shortest first, in this mode. I haven't seen it target urls with short words so much as just short urls first.