Forum Moderators: goodroi
basecamphq.com also provides "private" RSS feeds that anyone can theoretically access, but are meant for private consumption. There are probably other sites with this problem, and I expect it to become more common as RSS gets more pervasive.
Now my first gut reaction to this problem is to hack robots.txt to support a new designation. For example:
User-agent: *
Noindex: /rss
"Noindex" meaning: it is ok to programmatically retrieve data from the /rss directory, but NOT ok for a search engine to display any of the RSS feeds to people who aren't already subscribed.
Yes, this is a hack. Can you think of a better way? using robots.txt or something else?
I'd love to get buy-in from the web-based aggregators out there to make this an unofficial standard of sorts.
What do you think?
Apologies if this is confusing, this is not (for me) an easy problem to describe.
Phil