homepage Welcome to WebmasterWorld Guest from 54.211.230.186
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Microsoft / Bing Search Engine News
Forum Library, Charter, Moderators: mack

Bing Search Engine News Forum

    
Msnbot fails reverse DNS test
new ip block 65.54.165.XX fails reverse DNS lookup
Shurik

10+ Year Member



 
Msg#: 3520664 posted 4:14 am on Dec 5, 2007 (gmt 0)

msnbot now comes from 65.54.165.XX which resolves to by1sch4030215.phx.gbl instead of promised search.live.com

2007-12-05 00:02:54 W3SVC278140942 XX.XX.XX.XX GET /Index.aspx - 80 - 65.54.165.42 HTTP/1.0 msnbot/1.0+(+http://search.msn.com/msnbot.htm) - - www.mysite.com 301 0 0 320 260 109

This breaks reverse DNS lookup on my site (and many other sites) with all the nasty consequences.
What pisses me off most is that they themselves recommended to use reverse DNS to identify their bot!

[blogs.msdn.com...]

Very sloppy MSN, very sloppy.

 

jdMorgan

WebmasterWorld Senior Member jdmorgan us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 3520664 posted 4:49 am on Dec 5, 2007 (gmt 0)

There are actually two PTR records for addresses in that range; If your server checks only the first (which appears to be a CNAME), then it may fail rDNS verification. The second record points to hotmail.com

Based on this and what I've seen of their robots.txt and on-page robots meta-tag handling, I have to agree with your assessment.

Jim

[edited by: jdMorgan at 4:49 am (utc) on Dec. 5, 2007]

incrediBILL

WebmasterWorld Administrator incredibill us a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month



 
Msg#: 3520664 posted 5:30 am on Dec 6, 2007 (gmt 0)

Now you know why I use a combination of methods.

Cached reverse DNS lookups for a quick test for bot validity and if that fails I check to see if it at least belongs to a range owned by the company associated with the bot and do this for Google, Yahoo, Microsoft, Ask, Gigablast, etc.

[edited by: incrediBILL at 5:30 am (utc) on Dec. 6, 2007]

msndude

10+ Year Member



 
Msg#: 3520664 posted 6:56 am on Dec 6, 2007 (gmt 0)

Thanks Shurik for detecting this configuration issue on few servers, we will fix that.

Thanks again,
Msndude

Fish_Texas

5+ Year Member



 
Msg#: 3520664 posted 2:15 pm on Dec 6, 2007 (gmt 0)

Shurik wrote: This breaks reverse DNS lookup on my site (and many other sites) with all the nasty consequences.

Shurik: Excuse my lack of knowledge in this area, but does this have something to do with Slow Crawling, No Indexing or NO RANKING and NOT SHOWING in serps?

My MSN traffic stopped cold Oct 1st at 10 am Pacific Time when they switched on the new Algo.

Will this "fix" that MSNDude speaks of, address the issue and return the many lost sites to the serps?

Many of us are at a loss as to why we are not in the serps with this new Algo.

Thanks in advance for a post.
Fish Texas

thetrasher

5+ Year Member



 
Msg#: 3520664 posted 10:32 pm on Mar 10, 2008 (gmt 0)

65.55.104.29 - - [29/Feb/2008:02:46:46 +0100] "GET / HTTP/1.1" 403 *** "-" "msnbot/1.1 (+http://search.msn.com/msnbot.htm)"
65.55.104.173 - - [07/Mar/2008:03:53:54 +0100] "GET / HTTP/1.1" 403 *** "-" "msnbot/1.1 (+http://search.msn.com/msnbot.htm)"
65.54.165.47 - - [08/Mar/2008:14:05:33 +0100] "GET /robots.txt HTTP/1.1" 200 *** "-" "msnbot/1.1 (+http://search.msn.com/msnbot.htm)"

65.55.104.29 = bl1sch2033920.phx.gbl
65.55.104.173 = bl1sch2042520.phx.gbl
65.54.165.47 = by1sch4030220.phx.gbl

Shurik

10+ Year Member



 
Msg#: 3520664 posted 4:03 pm on Mar 11, 2008 (gmt 0)

Four month passed and the problem is still not fixed.
Shame on Microsoft.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Microsoft / Bing Search Engine News
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved