Forum Moderators: martinibuster

Message Too Old, No Replies

blocked urls by robots txt. but they aren't

         

Kickedout

3:33 pm on Jun 24, 2010 (gmt 0)

10+ Year Member



I find a ton of "blocked urls" in adsense report saying they are blocked by robots.txt which actually has Google media partners allowing....

They are mostly kinda
http:/ / www. google. cl/ imgres? imgurl= http:/ /example.com/bla

I also have bing [bing.com...]

I manually checked on those image searchers and I see my images.

What I do have is a .htaccess antihotlinking code like this
and to me is working ok..

#anti-hotlinking 
RewriteEngine on
RewriteCond %{HTTP_REFERER} .
RewriteCond %{HTTP_REFERER} !^http://(www\.)?example\.com/? [NC]
RewriteCond %{HTTP_REFERER} !(google|msn|yahoo|bing|picsearch|altavista|ditto|ask|live|aol|terra|alltheweb|lycos|pixsy|mamma|metacrawler|ixquick|grippo)\. [NC]
RewriteCond %{HTTP_REFERER} !search\?q=cache [NC]
RewriteRule .*\.(gif|jpg|jpeg|bmp|png)$ http://exmple.jpg [R,L]


Are those "blocked urls" reported correct?....
Am I missing something?

nomis5

7:57 pm on Jun 24, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Same for me, for a couple of months. I have no htaces either. Posted the same a couple of months ago and never got an answer I could understand. Hope you get an answer.

Kickedout

4:59 pm on Jun 29, 2010 (gmt 0)

10+ Year Member



I don't actually...

IanCP

3:10 am on Jun 30, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



"I find a ton of "blocked urls" in adsense report saying they are blocked by robots.txt"

I've had the same for a very long time. They are all translated pages though. I suspect it might actually have something to do with my frame busting script.

nomis5

5:01 am on Jun 30, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Mine are also translated pages but I have no scripts which would cause any problem. In fact, I don't have any scripts of my own at all because I don't know how to write them! No databases either.