Using content from content publishers

How would someone go about creating a comparison shopping site for example

[cheemo]

Although I am not trying to make a comparison shopping site it would help me to understand whats involved so I can apply that knowledge towards a site I am thinking of doing.

From what I would expect, stores like buy.com, etc would post somewhere an xml file containing products and prices, then other sites could use that xml file and create their own xsl files to display the other stores content on their own site.

Am I right at all or way off base? Do these XML files exist and if so how would I be able to access them? Sorry if I'm way off, but I've spent the last couple of hours looking into this and I can't find anything. Any info would be appreciated. Thanks.

[DaveAtIFG]

Welcome to WebmasterWorld cheemo! Since I have no experience in this area, I'd probably go look at the source for as many sites as possible to see how THEY did it!

Hopefully, someone with some experience in this area will offer a better suggestion.

[edited by: DaveAtIFG at 8:48 pm (utc) on Aug. 6, 2002]

[richlowe]

I am not sure if this helps, but check [syndic8.com...] - it might have what you need or point you in the right direction. Basically, it's a collection of RSS feeds which you can access from your site to get content from other sites.

Don't know what RSS is? Check that site and it will be explained.

Richard Lowe

[Nick_W]

Amazon do an zml thing, but it will depend from merchant/content proveder ......

Nick

[cheemo]

Thanks for the responeses thus far.

However I am still trying to figure out how this works though. I had thought that sites may be using an rss file to broadcast their content but for the comparison shopping site I am not sure (the syndic8 site is good for a lot of smaller sites but I would also like to know where content from the big boys comes from). For example a site like buy.com, found in all the comparison shopping sites where is it's rss file located and if there is one wouldn't it have to be massive, dispalying all their inventory?

As for going to the comparison shopping sites and seeing how they have done it dosen't really help because they just spit out normal looking html to the users. What I want to know is what is going on behind the scenes to arrive at the nice looking website. Where does all the content come from? and how is it structured?

[buckworks]

Many merchants provide plain-vanilla individual product links for affiliates to use; some provide full-fledged data feeds which affiliate sites can manipulate with their own scripts. You're right that some of them are large; for example, if you used the entire data feed from [an unnamed merchant selling collectibles] you'd end up with 36,000 pages.

Merchants are rightfully cautious about who is allowed to use a datafeed. You have to either have a strong existing sales record or else be prepared to pay for them. Example: at Performics, if you don't already have sales over $20,000 the fee is $2500 for access to merchant datafeeds.

[Hawkgirl]

Or, in my particular case, we get data from suppliers individually and then enter them into a database by hand.

Too many different formats from too many providers for us to be able to do things automagically.

[Brontojoris]

I don't really think it would be in Buy.com's business interest to provide an rss feed or whatever, so that you can create a 'Comparison Shopping Site'

It would immediately put them into a price war with every other vendor out there on the net.

Therefore I think you are probably going to be limited to using Affiliate linking (Linkshare, Commission Junction etc), or by screen scraping sites, and collecting the data yourself.

It's a real bummer for your 'Screen Scraping software, when the site layout changes, and don't expect too many checks from your Affiliate links.