Forum Moderators: martinibuster

Message Too Old, No Replies

Adsense blocked url by robots.txt issue

Site diagnostic

         

silverbytes

1:47 am on Jan 31, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Under site diagnostic I see some entries as blocked url by robots.txt, however I have
User-agent: Mediapartners-Google*
Disallow:

Which allows the bot crawl everything. One of these
are http:/ / images. search. yahoo. com/ search/ images/ and inside that I have several entries like this:


view? back= http%3A%2F%2Fimages. search. yahoo. com%2Fsearch%2Fimages%3Fp%3Dthe%2Bsimpsons%26imgsz%3Dall%26js%3D1%26ni%3D20%26ei%3DUTF-8%26vm%3Dr%26fr%3Dyfp-t-501%26b%3D281&w= 1024&h= 768&imgurl= www. mysite. com.%2Fsimpsons-photos%2Fwallpapers%2Fthe-simpsons-homer. jpg&rurl= http%3A%2F%2Fwww. mysite. com. ar%2Fwallpapers%2Fthe-simpsons-homer. htm&size= 48. 4kB&name= the-simpsons-homer. jpg&p= the+simpsons&type= jpeg&no= 289&tt= 252,991&oid= 102814dd35c3818e&ei= UTF-8

Is there anything I can change in robots.txt to avoid that? And what exactly does that mean?
Is there anything to change in .htaccess maybe?

OutdoorWebcams

9:47 am on Jan 31, 2008 (gmt 0)

10+ Year Member



This is nothing to worry about, I got some of those entries also.

It simply says that someone was viewing your page on Yahoo image search (and Yahoo then frames your page). The valid robots.txt in this case for the Mediabot is images.search.yahoo.com/robots.txt and that one prohibits spidering.

So you can't do anything to avoid that, only Yahoo could ;-)

silverbytes

12:20 pm on Jan 31, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Fast and clear. Great answer, thanks!