How a Price Comparison Search Engine works

[unbeatabletechnology]

I have some questions as to how the Price Comparison Search Engine Works. First off what languages, technical knowledge, and programming skills are necessary to create a search engine. Secondly how does the site update the information from the merchants. How do they input information about the products and pictures from the merchants? Also what other information would be necessary.

-Mike

[jmccormac]

I have some questions as to how the Price Comparison Search Engine Works. First off what languages, technical knowledge, and programming skills are necessary to create a search engine.

You need someone capable of writing search spiders. You also need someone to design and administer the database. You need someone to liase with the owners of the sites being spiders. It would be unlikely to find all these skillsets in the same person.

Secondly how does the site update the information from the merchants. How do they input information about the products and pictures from the merchants?

The spiders collect the data from the sites and parse it. It is then ready to be inserted into the database and crunched. The database then publishes out the comparisons to the web.

Also what other information would be necessary.

A business plan that involves making money.

Regards...jmcc

[CSE_Monkey]

Actually most comparsion shopping sites (Pricegrabber, NexTag, Shopping.com, Bizrate, etc.) aggregate merchant datafeeds which they then turn into an inverted index which is then searched using lucene and/or other analysis for relevancy.

Throw in a sprinkling of CTR and CPC for weighting puposes and there you have it.

;-)

Cheers!

[vik_c]

I believe almost all of them use XML. They get datafeeds from the merchants in the structure and format they want or get datafeeds and then convert them to a structure/ format they can use. Depends on who calls the shots. Large search engines like Shopping.com have a certain specific format/ structure in which the feed has to be made available by the merchant. They make money selling PPC advertising. With a couple of big deals in this industry I'm positive that we'll see many other new ones mushroom over the next few months which are possibly being worked on right now.

[ganrel12]

Do you need to pay websites or work out a partnership in order to spider them. Or can you just spider sites and get key price information for products?

Well,google doesnt ask you before it spiders your site,so spidering must be legal,right? Or is it?

[abbeyvet]

Spidering may be legal - though of course sites can block you. What you do with the information you acquire that way however is governed by all manner of laws.