|Google Shopping API|
bringing on the image scrapers
| 7:27 am on Dec 7, 2012 (gmt 0)|
Since we signed up with Google shopping's data feed, we've had a barrage of image bot thiefs pounding our site. They aren't walking the site, they're using the Google API.. they have to be since the image sizes that we have in the feed and the location isn't accessible publicly from our website.
Just though I'd give you guys a heads up as to what appears to be going on. If your competitor has use of the API and you don't set in google merchant to not allow use of your data in the API then be prepared to defend.
| 12:11 pm on Dec 7, 2012 (gmt 0)|
How are they pounding your site, if the location isn't accessible publicly?
Your best bet is to watermark every image, either with your site name or product description, then add your copyright to the EXIF data. I use an asp.net handler to do this on the fly, so I can store the raw images without watermarks.
| 1:01 pm on Dec 7, 2012 (gmt 0)|
Yes, i'd be interested to know how they are doing that.
Clearly, if google's API is being misused, Google need to stamp on that quickly.
| 3:23 pm on Dec 7, 2012 (gmt 0)|
Some where pounding it through proxy services and some once I contacted them regarding their content and image theft started not sending the referral string and common user agents. I spent about 12 hours combing through our logs.
It's a major MAJOR flaw. Competitors should ONLY be able to MAYBE see pricing comparisons and even then it really puts smaller companies at a serious disadvantage and is borderline corporate espionage.
Google does not clearly state in the Google Shopping interface that competitors can get ALL YOUR INFO including titles, pricing links to your products and links to your images through a publicly accessible API.
Here's some of what I've been going through:
The ones I'm dealing with are not using that user agent any more.
When using the APIs, the following prohibitions apply:
You will not sublicense an API for use by a third party. Consequently, you will not create an API Client that functions substantially the same as the APIs and offer it for use by third parties.
You will not perform an action with the intent of introducing to Google products and services any viruses, worms, defects, Trojan horses, malware or any items of a destructive nature.
You will not defame, abuse, harass, stalk or threaten others.
You will not interfere with or disrupt the APIs or the servers or networks providing the APIs.
You will not promote or facilitate unlawful online gambling or disruptive commercial messages or advertisements.
You will not reverse engineer or attempt to extract the source code from any API or any related software, except to the extent that this restriction is expressly prohibited by applicable law.
Certain developer credentials are, by their nature, considered confidential. You will not disclose such confidential credentials to any third party except your agent(s) using such information solely on your behalf in accordance with these terms and under a written duty of confidentiality.
Our communications to you may contain Google confidential information. If you receive any materials or communications that are clearly confidential or marked confidential, then you will not disclose the Google confidential information to any third party without Google's prior written consent.
| 4:06 pm on Dec 7, 2012 (gmt 0)|
Ok, I've got their protocol on the screen in front of me, and I don't see a "download my competitors products" API call.
Can you clarify what API you think is a major, major flaw?
| 5:04 am on Dec 9, 2012 (gmt 0)|
It's not a download api call .... but you can write your own program that makes calls to the api, which gets product data of whatever query you want, you can even get the links to the images that anyone is using for their ads.
From that you can run another program or your own little home spun bot of choice to go fetch those images as you see fit.
And there's no way to limit who really gets the feed... you cant say sitex can use my data in their app, but not siteY.
Either you approve it for apps outside of google shopping or you dont it's black and white.