I'm in need of a method (software, script, etc) to gather and report pricing information for a list of products. The list of products would include the Widgets that I sell. The database I'd like to draw from is Froogle (or Google Base), via HTML parsing.
Example: I sell thousands of different-style Widgets on my web site. I'd like to generate a report showing my price for each widget, along-side the price for that same Widget from other eCommerce sites, as submitted to, and reported by, Froogle.
I'd like to filter the results to eliminate pricing from auctions and small-time sites, and only report (for example) my five main competitors.
Ideally, I'd like the data gathering to take place at a continuous, slow pace, such that pricing information for each of my 1000's of Widgets is updated once every two weeks (e.g., query Froogle once every five minutes).
Since the result-set returned by Froogle is not precise, I'd like to mark specific results as "ignore", and have them excluded from future reporting.
I'm currently performing this price gathering manually, i.e., typing at the keyboard. I'm simply looking for a way to automate that task.
Thanks in advance for your ideas and comments.