Forum Moderators: open

Message Too Old, No Replies

How to get Yahoo feed bots to stay away

         

Angonasec

1:26 pm on Nov 18, 2013 (gmt 0)



We stopped our daily feed a year ago, after we discovered it was being auto-scraped and published on Blogspot without permission.

Most feed bots stopped calling when they realised the feed wasn't being updated, but Yahoo's feed bots continue to hammer even when blocked. Thousands of hits a day for a year now.

Any ideas how to get Y! to stop their bot at source?

I've not tried contacting them directly yet.

98.139.134.97 - - [17/Nov/2013] "GET /feed.xml HTTP/1.1" 403 304 "-" "YahooCacheSystem"
67.195.51.215 - - [17/Nov/2013] "GET /feed.xml HTTP/1.1" 403 304 "-" "YahooCacheSystem"

Such a waste of their resources and ours.

Angonasec

8:01 am on Nov 22, 2013 (gmt 0)



Not to worry, I finally got Yahoo!'s foolish bot to recognize a 410 and it appears to be having a lie-down.