Welcome to WebmasterWorld Guest from 174.129.151.95

Forum Moderators: bill & werty

Message Too Old, No Replies

How to use RSS to scan 14000 news sources

   
12:59 am on Apr 9, 2004 (gmt 0)

WebmasterWorld Administrator bill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



I'm always reading how some of the RSS big-wigs are able to scan an insane number of news sources on a daily basis. Do any of you do this? Do you have any pointers for how to amass such a collection of sources? What's a good strategy for filtering all of the information?

I find that it's tough to find sites I would consider reading regullarly in the first place...let alone a large enough collection so that I could have my finger on the pulse of the Internet...or whatever it is these gurus do...

6:21 pm on Apr 10, 2004 (gmt 0)

10+ Year Member



It's fairly easy to accomplish--moreso than one would think from the get go. We do it consistently without any problems, but the feeds are not for any other use than our own. Perhaps this is where we save tremendous resources.

I recommend having your own news application custom built rather than going with a service (such as the one in UK). It's cheaper in the long run, and you have total control.

However, I do not think many SEO's are onto this concept yet, but as with anything effective, they will in due time.

Good luck,

CF

3:05 am on Apr 11, 2004 (gmt 0)

WebmasterWorld Administrator bill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



I'm talking about people like Robert Scoble, Joi Ito, Nick Bradbury and the like. They all seem to be using commercially available software to accomplish this.
4:31 am on Apr 13, 2004 (gmt 0)

10+ Year Member



There are large collections of RSS sources around. There are lots of free and commercial tools available.

Since the rules of this board (I just got here) seem to strongly discourage any type of self-promotion or linking, I'm not sure how to go about sharing this information.

8:02 am on Apr 13, 2004 (gmt 0)

WebmasterWorld Administrator bill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Welcome to WebmasterWorld jeffbarr

I'm familiar with a lot of the tools and resource sites, but still wondering how others manage this. For example do you just download other people's opml files or is it a grueling process of accumulating thousands of sources one by one. It seems to me that just having a ton of sources really isn't good. You'd want to qualify those sources somehow, wouldn't you?

7:34 pm on Apr 24, 2004 (gmt 0)

10+ Year Member



Thanks for the welcome, Bill.

Some of the feeds are classified using DMOZ or other categorization systems. This simplifies the process. Right now, finding feeds is a manual process, but there are directory sites (PM me for mine) that make use of this info.

3:49 am on May 10, 2004 (gmt 0)

10+ Year Member



Hi Bill,

You can use software to scan usenet groups and convert those into RSS feeds. Scraping sites for content isn't that difficult either, all you need is to spider a list of sites on a regular basis really.

[edited by: Woz at 6:56 am (utc) on May 12, 2004]
[edit reason] Tidying up [/edit]

5:09 am on May 12, 2004 (gmt 0)

WebmasterWorld Administrator bill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Welcome to WebmasterWorld Lionhardt

I guess my conundrum is gathering lists of sites to scan. I'm probably like a lot of you; if it's not mentioned on WebmasterWorld then it probably isn't newsworthy, right? ;) I've been alerted to more major news events via Foo than any other source in recent years...I guess I'll have to keep plugging away on my list then. There don't seem to be many shortcuts to building a quality resource...but there rarely are...