Welcome to WebmasterWorld Guest from 184.73.112.180

Forum Moderators: Ocean10000 & incrediBILL

Should I block msnbot-media

   
10:20 am on Apr 9, 2014 (gmt 0)

5+ Year Member



I've noticed it's blocked on this default list:
[github.com...]

Is this really recommended? could it hurt my indexing and ranking with bing?
5:54 pm on Apr 9, 2014 (gmt 0)

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



I don't block msnbot-media. It is used for several purposes including the Bing Image Search. If you don't want your images indexed, disallow your image directories in robots.txt or use an X-Robots-Tag in your directory's htaccess.

I would never use someone else's block list, ever.
7:35 pm on Apr 9, 2014 (gmt 0)

WebmasterWorld Senior Member dstiles is a WebmasterWorld Top Contributor of All Time 5+ Year Member



I block the media bot (also from G and yandex). Seems to make little real difference apart from image indexing, which is irrelevant for my sites.
8:01 pm on Apr 9, 2014 (gmt 0)

WebmasterWorld Administrator 5+ Year Member Top Contributors Of The Month



I have to agree with dstiles, I also block the msnbot-media bot. It is a personal preference based on their observed behavior. They never 304 for images they already have and they seem to ignore no-index header tags for images used in css, have a sweet tooth for old buttons images kept for some legacy pages - and other useless image files. I recently noticed ignoring of robots.txt so they are blocked via UA. I don't have issues with Google-Image and other allowed bots. Their sprees of requesting hundreds of non-existing files didn't help.

I also agree with keyplyr - it is best to observe the access logs for your own site and not use block lists found online. Some are quite old, full of things that never visit and busy the server just a bit more as they check visitors for everything you have in the list. Split seconds can add up in load time.
8:26 pm on Apr 9, 2014 (gmt 0)

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Well if you block these media bots you block your link imagery from showing up in most of their social media as well as imagery for their maps & videos in the SERP. Personally I find this quite valuable, accounting for measurable traffic.
 

Featured Threads

Hot Threads This Week

Hot Threads This Month