Price comparison affiliate sites

How is it done technically?

[arikgub]

Sorry, may be it's a dumb question but I'm a newbie and trying to understand how the stuff works.

There are many affiliate sites that allow you to search for a particular product in many stores and return you availability + price info + an affiliate link that leads to the product page. Technically, when the user submits the search form, where do price comparison sites take the price info from? Few possibilities:

(1) The search form activates CGI script that reads the price info in "real-time" from the merchant site and then generates HTML with the price to the user. (pros: always updated info, cons: takes time to gather all data, slow response)

(2) All the price data is stored locally on the affiliate site in some kind of database. (pros: fast response, cons: info may be outdated)

How is it actually done? If the option (2) is used how the data coherency with the merchant site is kept? Merchant can update the price, or the product may be out of stock....

I'll appreciate your replies.

[SeanW]

I do it with option #2 using datafeeds and scrapers to periodically refresh the data. There's also a great deal of work in parsing the information so that you can say that vendorA's product #*$! is the same as vendorB's product.

Sean

[arikgub]

Thanks for reply, Sean.

I'm building a price comparison site now and trying to automize its maintenance as much as possible. I prefer "launch it and forget about it" approach but it doesn't seem feasible in this case :-)

The problem is keeping the info up-to-date and also the one that you mentioned, correct identification of the same product offered by different vendors. All this requires manual intervention....

[macz_g]

I have noticed that most merchants will not allow you to use their feed in this way. It would violate the affiliate agreement.

Has anyone ran into this kind of problem or know ways around it?

[vfilip]

I have the same problem . I am in the travel industry and I have seen some comparison sites that a user asks for a specific city hotels (with some specifications like arrival and departure dates ) and the site returns prices from several suppliers with short description , actual price , and a link in order to proceed with buying of it . That is real value to the end user . I predict that have high conversion rates . I am talking about sites like kelkoo and some other smaller.
I can understand that they built such a comparison engine with Realtime XML feeds asking the supplier in real time at backend and quoting in their site .
so the answer to your question is that you should get a realtime XML feed .
The strange thing when i conducted the suppliers to give me the realtime XML feed they were reluctant to give me . I don't know why . Has anybody such an experience?
I started once to build an engine for booking flights without this realtime XML feed and i reached the point where i constructed url queries (url queries to merchant sites ) showing to the user the links where he could find more info like , price , descr etc. Really very hard work trying to understand how the merchant works , what are the parameters passing , deep linking and so on.
But these comparison sites are far more better . They show actual price + descr + link to book .

If someone would give any feed would be very much appreciated. not XML feed :)