|Does Google reject automated ranking queries?|
The company I do SEO work for uses a popular commercial ranking/submission software program.
I get accurate ranking results from Google as well as the other engines I've selected. However, it seems that if I run too many ranking queries in too short a period of time, Google either gives incomplete results or no results at all.
For example, on Saturday I ran a ranking report for one of our clients. For some reason, the software quit on me late in the process (it takes about 13 hours for this particular client, as they have so many keywords to search for).
When I realized the program had quit, I started it up again, and ran another ranking report. This time, there were no results at all from Google. None.
I'm wondering if Google sees that the queries are coming from the same IP, or that it recognizes that the queries are automated. I'm also wondering if Google punishes this sort of activity.
Any comments much appreciated.
The cutoff is 1,000 queries a day -- and yes, the Terms of Service denies automated queries.
|No Automated Querying |
You may not send automated queries of any sort to Google's system without using any software which sends queries to Google to determine how a
express permission in advance from Google. Note that "sending automated
queries" includes, among other things:
website or webpage "ranks" on Google for various queries;
"meta-searching" Google; and
performing "offline" searches on Google.
Terms of Service [google.com]
Even with long delays built in between queries, Google will shut you off at 1,000 queries within 24 hours, as you've discovered. And if they feel you are abusing their service and they can connect that activity to a website, well...
|Quality guidelines - specific guidelines |
...Don't send automated queries to Google.
...If a site doesn't meet our quality guidelines, it may be blocked from the index.
Webmaster Guidelines [google.com]
You can use the Google API for automated queries. However, there is still the limit of 1000 queries/day.
I don't believe the 1000 queries per day story. In a normal SEO company with 50 employees that do 50 queries a day, the querycounter reaches at least 2500.
Our automated tools do 10.000 queries a day once a month. And we don't get blocked...
Okay but 13 hours of constant querying? :D
I mean this is a desktop application right?
You're basically doing queries from your computer's IP in the bulk.
Of course you'll get busted sooner or later.
And that it comes back afterwards is just a proof of tripping the good-faith too many queries filter that's on the user side.
But why don't you get some software/web app that uses API?
You could have more than one API key... ( or can you? )
...also what's the point of doing all this when there're things like server stats, Google analytics, Google webmaster tools ( i mean the query info, broken down by region, search type, displayed and clicked results )...? Checking your position with a few milestone phrases that would be most relevant would be enough i think... it gives you an idea whether you're trusted in that particular area or not, the rest is up to parameters you already should be aware of anyway ( on and off-site relevance ).
Unless you optimize for everything and check regularly whether you've won the lottery, but that doesn't work that way anymore. Or is your client THAT big of a player ;)
Well they certainly can block the IP from a Google search - that is a fact, I do not know how much traffic would annoy them - 1000 per day is way low I think and how do they cope with proxies?
I think it is totally untrue to say auto queries could harm a sites rankings - obviously this is way too easy for a competitor to kill another competitor's rankings.
I just ran a ranking report for a different client's website. There were no Google results for that site, either. So, it's not the specific website that's being punished, it's the IP.
Two weeks ago, I ran a ranking report with no problems from Google. It must have been the back-to-back queries that caused our IP to be blocked.
We can change the server IP, and see if that makes a difference.
The company I work for has been using this software for over two years, and there haven't been any problems.
As for why we don't use a different ranking software program, well, I'm not the boss. ;) He feels that clients should be presented with fancy reports with all sorts of graphics and so on.
For my own website, I'll just check a couple of dozen keywords and phrases by hand every month or so. And I look at my stats. As long as roughly 45% of my traffic is coming from Google, and the traffic isn't going down, I assume I'm still ranking very well.
The creator of the ranking program we've been using replied to me saying that, sometime in mid-November, Google changed the way it delivered search results, and his software program is not currently able to retrieve those results.
I tend to believe the guy, as his software has worked well for more than a couple of years.
Meanwhile, we downloaded a different ranking program onto the same server. Same server, same IP.
I selected 260 English language-only search engines, and selected 205 keywords and phrases.
It took 197 hours for this program to complete the task. Thank God I wasn't running this program locally, or I would have been stuck doing nothing for eight days while I waited.
My point, though, is that the queries were coming from the same IP.
So, there must be something that's changed with Google.
Any ideas what that might be?
Why do you think it took a week and a day?
It's surely not the bandwidth :P
I think it took eight days because the program is not very good.
Here's a weird thing, though. Last night I ran a different ranking query for a different client, using the old ranking software program. And it delivered results from Google.
I'm really getting confused.
Eight days doing massive querys?
IMHO the G must ban the abuser IP, because such activities is very resource intensive and costly for nothing, just to check rankings. Also why check who have thousands words/phrases combinations?
Only watch the traffic, if goes up is good. For real world only small niches can take advances to know when reached the limit (the first position).
I don't often make posts like this, so I'll apologise in advance. I'm seething.
The attitude of the 'SEO' company in question stinks, from their point of view it's OK to suck up others resources so they can massage the ego of their client/themselves. It's this type of disregard for others that leads to restrictions for those that play by the rules. OK, the Google search API limit is restrictive for some applications (including the bespoke software we have written) but it's a daily limit so you just have to accept that things may take longer than you wish.
Google clearly state their position on automated queries, and just because you don't get blocked too often does not mean you are in the right.
While I'm on a rant, what makes anyone think that there are 200 search engines that matter? Reducing that to a top 10 or 20 would surely be sensible if it's going to take a week off the running time.
inbound, I do not disagree with you at all.
For my purposes, I'm only interested in three or maybe four SE's. And, really, my focus has always been on Google. For the most part, if I can make it on Google, I've likely made it on all of the other engines that really count.
But, the client and the boss disagree.
I let this poorly-performing program run so that I could explain to my boss why we shouldn't use it. 197 hours doing ranking analysis seriously hampers the number of clients we can serve. Call it a protest action, if you will. He's now in my camp.
For 205 keywords and phrases, I could do a manual search on Google in less than an hour and record the results. That's all I think the client needs.
My boss asked me to work for his company because I've been very successful at getting first-page results for the websites I've worked on, including one of my own. While I obviously paid attention to SERP's, I never ran an automated ranking program. Didn't see the need to.
Consider this my little rant: I now spend more time generating all sorts of reports and giving clients all the razzle-dazzle than I spend doing real SEO work. To my way of thinking, I should just be able to direct them to their monthly stats report, show them that their traffic is increasing, show them where the traffic is coming from, and hope for a "thank you."
I'm always afraid of ticking off Google, and having that software running for 197 hours really worried me.
I've always wondered...
Do most of the seos get paid for the number of positions the client climbs up? I think it'd be pretty dumb but... is this right?
If so, i can understand this query-happy attitude of the company, but... no. Not even then :I
I just don't get it, it's about two things actually.
The people who find you, and whether they liked your site or not. Which... again... is so easy to check via GWT and analytics that not even that would justify this automated search mania.
Does your boss know that GWT shows not only clicks but displays on SERPs as well? Meaning if someone does a search, and the site shows up, GWT will tell you it's position even if nobody clicks the links.
whatever people say, i like anaytics and GWT...
What is GWT?
As for the reports and querying and all, I'm seeing in all sorts of industries a greater focus on the presentation than on product quality or service.
About two years ago, a designer asked me to sit down with a client of his who has had a website since 1997. The client was a medium-sized manufacturing company.
They'd had a well-known registry company doing their "SEO" work for them for two years. The client was paying $2000 a month. But the client's site wasn't ranking well at all. In fact, their rankings were terrible. They had paid $48,000 and some of their pages weren't even in the top 800 results on Google.
The big-name registry company knew that the client was going to be getting quotes, so then sent in a salesman who left behind a very slick, four-color, bound folder with charts and graphs and all sorts of stuff.
I presented them with a well-written four page proposal that outlined what I would do, when it would be done, when they could expect to see results, and
what the cost would be. (Cost was the same as what they'd been paying).
A month or so after the meeting, I gave the client a call to see if he'd made a decision.
He had. He decided to stay with the big-name registry company, but he signed up for their $3000 a month "premium" program.
There's no cure for stupidity...?
btw. GWT is Google Webmaster Tools.