| This 31 message thread spans 2 pages: < < 31 ( 1 [2] ) | |
| Now seeing Bingbot
Testing the waters a few days early |
jdMorgan
msg:4208509 | 3:12 am on Sep 29, 2010 (gmt 0) |
Spotted in the logs today.
207.46.195.227 Tue Sep 28 20:24:15 2010 "GET /widgets.html HTTP/1.1" 200 4321 "-" "Mozilla/5.0 (compatible; bingbot/2.0 +http://www.bing.com/bingbot.htm)" Connection="Keep-Alive" Accept="*/*" Accept-Encoding="gzip, deflate"
Jim
|
Pfui
msg:4229278 | 5:59 pm on Nov 11, 2010 (gmt 0) |
Still hitting bare (no rDNS), but at least asked for robots.txt this time:
157.55.16.231
Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
robots.txt? Yes
Hmm. Bare MSN IPs get the standard disallow version of robots.txt, the official .search.msn.com/msnbot/bingbot bots don't. Yet another IP/UA variation/combination to code for? MSN, Google 's new Web Preview UA [webmasterworld.com...] -- it's never-ending, I tellya.
|
| This 31 message thread spans 2 pages: < < 31 ( 1 [2] ) |
|
|
|
