Welcome to WebmasterWorld Guest from 54.161.178.52

Forum Moderators: bakedjake

Message Too Old, No Replies

How a Price Comparison Search Engine works

     
6:56 pm on Jun 22, 2005 (gmt 0)

New User

10+ Year Member

joined:Sept 16, 2004
posts:27
votes: 0


I have some questions as to how the Price Comparison Search Engine Works. First off what languages, technical knowledge, and programming skills are necessary to create a search engine. Secondly how does the site update the information from the merchants. How do they input information about the products and pictures from the merchants? Also what other information would be necessary.

-Mike

6:38 pm on June 23, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Aug 30, 2002
posts:2536
votes: 47


I have some questions as to how the Price Comparison Search Engine Works. First off what languages, technical knowledge, and programming skills are necessary to create a search engine.
You need someone capable of writing search spiders. You also need someone to design and administer the database. You need someone to liase with the owners of the sites being spiders. It would be unlikely to find all these skillsets in the same person.

Secondly how does the site update the information from the merchants. How do they input information about the products and pictures from the merchants?
The spiders collect the data from the sites and parse it. It is then ready to be inserted into the database and crunched. The database then publishes out the comparisons to the web.

Also what other information would be necessary.
A business plan that involves making money.

Regards...jmcc

11:24 pm on June 23, 2005 (gmt 0)

New User

10+ Year Member

joined:Oct 22, 2004
posts:27
votes: 0


Actually most comparsion shopping sites (Pricegrabber, NexTag, Shopping.com, Bizrate, etc.) aggregate merchant datafeeds which they then turn into an inverted index which is then searched using lucene and/or other analysis for relevancy.

Throw in a sprinkling of CTR and CPC for weighting puposes and there you have it.

;-)

Cheers!

12:23 pm on June 27, 2005 (gmt 0)

Preferred Member

10+ Year Member

joined:Aug 13, 2002
posts:408
votes: 0


I believe almost all of them use XML. They get datafeeds from the merchants in the structure and format they want or get datafeeds and then convert them to a structure/ format they can use. Depends on who calls the shots. Large search engines like Shopping.com have a certain specific format/ structure in which the feed has to be made available by the merchant. They make money selling PPC advertising. With a couple of big deals in this industry I'm positive that we'll see many other new ones mushroom over the next few months which are possibly being worked on right now.
4:39 pm on July 4, 2005 (gmt 0)

New User

10+ Year Member

joined:Feb 21, 2004
posts:17
votes: 0


Do you need to pay websites or work out a partnership in order to spider them. Or can you just spider sites and get key price information for products?

Well,google doesnt ask you before it spiders your site,so spidering must be legal,right? Or is it?

4:48 pm on July 4, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Aug 29, 2003
posts:944
votes: 0


Spidering may be legal - though of course sites can block you. What you do with the information you acquire that way however is governed by all manner of laws.
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members