Welcome to WebmasterWorld Guest from

Forum Moderators: bill & werty

Message Too Old, No Replies

Splicing and storing multiple RSS feeds

3:20 pm on Mar 13, 2006 (gmt 0)

New User

10+ Year Member

joined:July 15, 2004
votes: 0

I syndicate (with permisison) a number of feeds (currently at 40) on a central hub site.

I currently use CaRP Evolution, storing data in a mySQL database, and using its caching functions. I update feeds once an hour, due to ISP restriction on cron jobs.

I'm wondering if there's a more efficient way of doing this. CaRP seems pretty good though sometimes feeds seem to stop updating for a few days, then start again, even though they're valid markup, and I wonder if it's a server issue or trying to collect too many feeds at once.

I currently have about 5 cron jobs running each hour that collect about 8 feeds each. This number is likely to increase over time.

Some pages use CaRP directly to display feeds, others rely on the mySQL database it maintains.

Any thoughts on the most efficient way that is scalable and doesn't place strain on the server or make my ISP mad? :)

I'll try external services (for example, I was looking at whether Bloglines could export a single feed from multiples) if they work well and reliably.

2:28 pm on Mar 23, 2006 (gmt 0)

New User

5+ Year Member

joined:Feb 16, 2006
votes: 0

Why manually update feeds with a cron job? Why not just grab them live using magpie?
8:44 am on Mar 24, 2006 (gmt 0)

Junior Member

10+ Year Member

joined:July 26, 2005
votes: 0

Sorry, off topic question but...

with permisison

Is permission required? The way I am doing this is as follows. If someone provides a feed on their site and they do not say commercial uses are not permitted, I see it as fit to be used.

I would like opinions on if this is a fair way to look at it. Of course, if someone requests for their feed to be removed, I would promptly do it.

My feeling is that unless there is a mention of 'no commercial use', the feed is free to use. Thought please (and sorry to go off topic).