Forum Moderators: phranque

Message Too Old, No Replies

Rss Threat?

         

keyplyr

7:21 am on Nov 17, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month




Lately I've noticed log activity showing several UAs associated with RSS: rss_reader , AvantGoRSS and Feed::Find to name a few. They are requesting only html files, no scripts, css or images. I'm wondering what to think about these UAs and have banned them until further consideration.

I do not offer RSS feeds. I really don't understand much about the subject except that RSS seems to be gaining popularity. A few mobile phone ISPs are now using it as a streamlined way to get web data: stock reports, weather, sports, etc. but none of these pertain to my site. My site is too table and graphic dependent to accommodate mobile devices anyway and I've not bothered to develop an XML/WAP version for this reason.

Can RSS crawlers "capture" my articles, content or anything else from my webpages and feed them to RSS services for distribution? Can my content be scraped by this method? Thanks.

bill

7:41 am on Nov 17, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



A lot of readers have simple viewers incorporated in them. Thus you can see the actual webpage associated with the feed. If some feed or feed's webpage linked to you then it might be possible that someone is surfing thru to your site. If someone has blogged about your site this is one thing I'd consider.

However, the fact that they're not requesting CSS or images is troubling. You might just have some scrappers with a renamed UA looking for content.

keyplyr

9:07 am on Nov 17, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



...not requesting CSS or images is troubling. You might just have some scrappers with a renamed UA looking for content

Pretty sure the HTML pages are just just being indexed. The UA "AvantGoRSS" belongs to AvantGo dot com, a service provider for mobile phones/appliances. My concern is for RSS in general being a threat to content saftey or not. Should I allow access by these UAs since I do not offer RSS feeds? And yes, I often see various blogs as referrers.