Forum Moderators: martinibuster

Message Too Old, No Replies

Mediabot claiming bogus robots.txt entries

I don't get it.

         

mattg3

1:13 pm on May 6, 2007 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



When I look in my adsense account under Website Diagnosis it claims that robots.txt restricts pages that aren't restricted. The collection is completely random.

The equivalent in sitemaps makes no such mistakes. Odd very odd. The whole collection starts on the 27.4. up to today.

I haven't restricted Media nor Googlebot from any of those pages and checked there are no other bogus robots.txt anywhere on my server. The directory that is supossed to have a very small percentage of all my files excluded isn't actually in the robots.txt which is unchanged since years.

I don't know what the hell it's doing, i emptied the robots.txt now, but I am 99.9% sure I didn't make a mistake.

icedowl

1:17 pm on May 6, 2007 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



You probably didn't make a mistake. I think they have or at least had an error. I've seen the same thing and know it's bogus. I just ignore it.