Page is a not externally linkable
- Yahoo
-- Yahoo Search Engine and Directory
---- Bye Bye Yahoo Slurp, Hello crawl.yahoo.net


Kurgano - 7:21 am on Jun 6, 2007 (gmt 0)


About time! Although I wish the multiple crawlers would share a cache too.

phpBB3 boards rely on labeling bots to remove session ID's from the uri's when they visit. I just looked in the "whos online" section of a buddies forum and of 29 users online over the past 30 minutes 4 are people and 25 are bots.

The board owner, has all bots share one account so the who's online section reads... Users online : googlebot, googlebot, googlebot etc.

Ironic huh?


Thread source:: http://www.webmasterworld.com/yahoo_search/3359251.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com