Google's Rank Modifying Patent for Spam Detection

[tedster]

Patent hawk Bill Slawski has published an article [seobythesea.com] about a newly granted Google patent - one that seems to explain a lot of the odd ranking behavior that has been reported in recent times. The patent is named, quite simply, Ranking documents [patft.uspto.gov]

As I understand it, the idea here is to identify, by algorithm, what looks like an attempt to manipulate rankings in a spammy way. The patent details things like keyword stuffing, invisible text, link-based manipulations and so on.

Rather than allow the rankings to respond immediately and directly to those changes, the patent explains a system that would change rankings in unexpected, counter-intuitive ways - while the rankings change from a first position through transition positions and to the final "target rank" position. In other words, significant changes in position continue to happen even though there is no change in page's ranking factors!

Bill explains:

During the transition from the old rank to the target rank, the transition rank might cause:
a time-based delay response,
a negative response,
a random response, and/or
an unexpected response

If these transitional ranking shifts are followed by what looks like a responsive action from the website, the URL or website which was previously only SUSPECTED as spam, might now be positively tagged as spam.

Read Bill's full article [seobythesea.com] (or the patent itself [patft.uspto.gov]) and see if it doesn't explain some of the odd behavior people have been reporting in recent months:

[Elsmarc]

@Leosghost

I understand. Whether local or not, when I saw the post where the person said they were dependent upon being #1 on page 1 and would have to go out of business if they were in slot 7 or 8 on page one, I knew for me it would be an impossible task if only for the reason that my "problem" site is too broad with too many topics to always hit page one for a given key word.

I have done well over the years so I have no real complaint. It may be time for me to move on to other things.

So it goes.

[Leosghost]

the person said they were dependent upon being #1 on page 1 and would have to go out of business if they were in slot 7 or 8 on page one,

What applies to them, may well not apply to you, nor I , nor others..

I knew for me it would be an impossible task if only for the reason that my "problem" site is too broad with too many topics to always hit page one for a given key word.

Realistically , you may well not have always been at number 1 for all your desired keywords, and until recently it did you no harm, ( I'm presuming the site you mention is the big one that I know of from your profile ? )..Sit it out..

Given that G is playing "random shuffle SERPS" to smoke out "the evil SEOs" that they don't like..if they are moving some of the pieces in the basket, by definition the others that they were not targeting may also move..collateral damage..you may rise back up again like walked on grass which was in their way when they decided to stomp on someone near you or that had your "profile"..

Don't "zig" because G "zagged"...in their eyes ..only the guilty "zag"..and only the real sinners run like a guy trying to avoid sniper fire..waiting till early December is a good idea..:)

[Elsmarc]

Yup - I have made some changes and I made out a "project plan" on 28 July. But all in all, nothing I am changing is what I consider "significant" other than to get into WMT and work with the URL parameters part in an effort to reduce duplicate thread titles, conflicts with t and p in URLs (etc.) and such, and yes I think the site is in my profile here (long time since I've even looked at my profile here...). It was down (as in dropping off a cliff on 25-28 July) but the last 2 weeks it appears to be coming back. But like I say - I see things longer term than many. I'll have all the changes complete within a week, 2 at most. From then on it's - Well, I'll just say "fate". I consider myself lucky that I don't depend upon income from it to live on.

If nothing else, I can say I had a relatively popular web site for 15 years. Up until a couple weeks ago it was always ranked at least around 70K to 85K or lower in Alexa (I know, but it's a "relative" indicator).

So - That's the scoop. I will say I was in total shock the first 2 weeks but I'm past that now.

The internet, it is a changin'... Like Dorothy not being in Kansas any more, I'm not in the internet of the 1990's and 2000 to 2005 any more.

[RegDCP]

@Leosghost

"Since G personalised SERPS ( whether one is "signed in" or not ), no one can say with 100% certainty that they are always at #1 all of the time"

I use an anonymous proxy when I am searching regarding SEO.

[aakk9999]

@RegDCP

I use an anonymous proxy when I am searching regarding SEO.

Yes, you may, but others are not, they will get "their" personalisation so even if you see non-personalised SERPs, others may not, hence your 'non-personalised' SERPs is not really representative.

[Leosghost]

^^^ what aakk9999 said :)

[RegDCP]

I hold that if your onpage SEO is done properly and your page generates a top result, then it will still be a top result despite personalization even if the said personalization is skewing the results.

[Shaddows]

I hold that if your onpage SEO is done properly and your page generates a top result, then it will still be a top result despite personalization even if the said personalization is skewing the results.

Then you totally misunderstand personalisation.

Personalisation is, at its best, matching query intents.

At it's most creepy, its about serving documents that match your "Grade Level"

At its worst, its about reinforcing your prejudices so you never have to encounter dissenting views (the infamous bubble)

[RegDCP]

But if the page, without personalization, makes the top of the charts, then even with personalization it will still be at the top, if you are looking for the same phrase.

[Shaddows]

No it won't.

Take a fictional example.
Say I was an American, and looked up "abortion"

Some people might see a load of emotional advice, support groups, maybe a clinic.

Someone else might see a load of religous advice, pro-life groups, maybe a church.

Someone else might see some scientific research or State and Federal legal definitions.

And for each of those searches, the results would be "right".

I pick a highly divisive topic to show how CONTENT and USER INTENT are the key to personal search. Please do not engage with the subject matter

[SevenCubed]

At it's most creepy, its about serving documents that match your "Grade Level"

At its worst, its about reinforcing your prejudices so you never have to encounter dissenting views (the infamous bubble)

Shaddows, Shaddows, Shaddows....YES! I am very aware that this is OT but what Shaddows has said, though it has been said in round about kind of ways here before, has been so precisely stated by Shaddows that having it re-enforced by this Filter Bubbles video on TED is very worth watching for our collective well-being... [ted.com...]

For those of us who live in ignorance we will be further dumbed-down.

[thegypsy]

Rushed for now, but to try in limited time to add a little context to this one;

A. Things like transition rank have been around since 03 (historic ranking patents)
B. Even this one, if implemented in some way, would have been in the wild since 09 (so not new).
C. Likely needs context; that a page/site has been flagged as spammy (consider how many changes Wikipedia has... Google sure loves them)

My sense of it is that this would be something applied to sites that have already tripped another spam signal.... Consider the processing of trying to run this on EVERY page on the web whenever changes are made. Doesn't fit with their 'need for speed' approach over the last few years.

This is yet another case, like LSI and the 'reasonable surfer' where the industry tends to jump on patent awards without the proper context to it all. This is NOT new. If you're SEO was working a month ago, it will still work just fine today.

I shall do my best to keep an eye on this thread..

Look forward to continuing the dialogue folks
.

[edited by: Robert_Charlton at 6:27 pm (utc) on Aug 30, 2012]

[RegDCP]

@ Shaddows
Yes you are 100% right.

However it still remains that a top ranking page will show in a personal search if relevance is confirmed.

Let's look at your fictional example.
If my page got a #1 for Abortion and it dealt with emotional advice and it turned up #1 for someone searching for legal advice, it would get a high bounce rate.

It does not matter if personal skews the results based on topic preference, the results outside the desired niche are not relevant.

When I search using a proxy, if my term turns up in #1, it will also be #1 in a personalized search.

[Leosghost]

When I search using a proxy, if my term turns up in #1, it will also be #1 in a personalized search.

No..it may be #1 in a proxy search ( and there are proxies and good proxies, data "leaks" through some ),* when you search without using a proxy, it may be #1, but only in your personalised search..

In someone else's personalised search ..it may not even be in the top 10..

*Sometimes even time of day, and other factors may alter the contents of even your personalised search..

[edited by: Leosghost at 6:30 pm (utc) on Aug 30, 2012]

[SevenCubed]

When I search using a proxy, if my term turns up in #1, it will also be #1 in a personalized search.

A proxy is only going to hide your IP Address (mostly). That still doesn't diguise your 57 other characteristics that google is sniffing. Kind of like a mailman not wanting to be bitten by a dog so he delivers his mail disguised as a clown -- the dog still knows what's going on and bites him anyway.

[aakk9999]

Shaddows:
Some people might see a load of emotional advice, support groups, maybe a clinic.

Someone else might see a load of religous advice, pro-life groups, maybe a church.

RegDCP:
Let's look at your fictional example.
If my page got a #1 for Abortion and it dealt with emotional advice and it turned up #1 for someone searching for legal advice, it would get a high bounce rate.

What you are missing, RegDCP is that you are looking at a query in isolation.

Your page may not turn up as #1 to someone searching legal advice. Google will make a decision based on past searches and their intent. So if past searches were to do with legal advice, then #1 may not be your page with emotional advice.

On the other hand, if your page is covering both, the legal and emotional issues and is strong in its #1 position, then it may still show as #1.

The thing is - you never know for sure.

I am using a tracking tool that for every hit shows me a page the user landed to, the query and the full referer URL. On many occassions it also shows the position the page was in Google SERPs. On many occasssions when I enter this exact referer from Google SERPs, I am not seeing the page on position the tracking tool said it was. Sometimes the page is not even in the first few pages of SERPs for the query, yet it was there for the visitor. On other occassions it is exactly where the refering query said it was. For selected queries I was following this closely and what I can say is that there are no rules.

You can also see this if you learn to read Queries from WMT, filtering it down to a country and "web" and then drilling down at the query details which shows you which page ranked and in what position.

[Robert Charlton]

Regarding thegypsy's post above... in his blog post today [searchnewscentral.com], David Harry (aka thegypsy) comments on the patent and some of the reaction it's been getting...

I personally get the sense that this would be something applied to sites that have already tripped another spam signal (there's a ton of them) as another bit of evidence prior to any type of manual action. I say that because if this was applied index-wide, the SERPs would actually be a massive state of flux. Think about it. It makes more sense as yet another layer of spam detection to avoid false positives.

Well worth reading in the context of this discussion.

[Leosghost]

What thegypsy missed there though is if they , Google, boost or lower or shuffle some sites in the pack , the others sites in the pack must also move ( up or down ), so it does not merely have the effects upon those whom Google intends to..

So it may only be run against already "flagged" sites or may be a "tipping point was reached, thus run" type algo, but there will be collateral effects, both boosts or damage, winners and losers..

[RegDCP]

What you are missing, RegDCP is that you are looking at a query in isolation.

Your page may not turn up as #1 to someone searching legal advice. Google will make a decision based on past searches and their intent. So if past searches were to do with legal advice, then #1 may not be your page with emotional advice.

On the other hand, if your page is covering both, the legal and emotional issues and is strong in its #1 position, then it may still show as #1.

This is what I am saying.

The superiority of the page in topic related results is the determining factor.

[RegDCP]

Collateral results will be minor.
If a page's terminal result is #2 and the patent ago decides to make it #9 on a temporary basis, then the #2 to #8 results have not been affected, while the #9 to infinity are seeing their move down as a natural result.

[Leosghost]

The superiority of the page in topic related results is the determining factor.

That is not what either Shaddows nor sevencubed, nor I am saying at all..

If a page's terminal result is #2 and the patent ago decides to make it #9 on a temporary basis

that is only one possible movement amongst many scenari available even for just the first 10 positions..( and the moves are never in isolation ) but you are not understanding any of this ..so hey.. carry on :)

[RegDCP]

I think it is you that is not understanding.

but there will be collateral effects, both boosts or damage, winners and losers..

How can there be collateral damage if the (patent's) results are lower than the final destination?

The positions above the temporary position are unaffected and the ones below are still only moved down as they would be if the page went straight to its terminal position.

Personalisation is, at its best, matching query intents.

SO IS SEARCH.
The single word example is not the best way to express this as search is rarely done on a single word (short tail) basis.

Your page may not turn up as #1 to someone searching legal advice. Google will make a decision based on past searches and their intent. So if past searches were to do with legal advice, then #1 may not be your page with emotional advice.

If your page is top, it will show in a personalized search using your preferences. If the search is not using your preferences then it will not show up, which is what you want also, as a slim relevance result will generate a bounce.

[Martin Ice Web]

I moved one domain to a new hoster because performance reasons. Same time i reworked some CSS styles. This is now 4 weeks, the site immediatly plumped in searches and did not come back till now, its even getting less traffic.

I redesigned a second site 3 days ago. This site has take a hughe hit yesterday. I will now wait 6 weeks and will not make any changes but will clearly add new content and will fix spelling errors.

[driller41]

It depends if this patent is active - it certainly makes a strong reason to leave a penguined/panda site alone and go off and start again on a new site.

[thegypsy]

right then....

How can there be collateral damage if the (patent's) results are lower than the final destination?

Keep in mind that it doesn't always move down. Would kinda work like this;

1. You're ranking in pos 20
2. You add links, mess with TITLE or other stuff
3. You changes would naturally take you to 5th
4. Google moves you to a transition rank of 10th OR 30th

It could move you up, just not as far as expected, causing you to do further changes to boost the rank.

Again, this really doesn't make sense in the larger context (index-wide) because we'd see a TON more flux that we do. Yes, I know there is a fair amount out there, but imagine if ANY time a page changed, got new links, that this was happening. The SERPs would be even whackier than they are. Bear in mind, this approach was toyed with in the 2003 historical ranking factors patents and even this patent was 2010 (so in play 09 at least).

So, to my thinking, this might be something that is triggered once a site has been in some way identified as potentially spammy already. To get a sense of the myriad of ways they look at webspam search out this other recent post over on SNC:

[google.com...]

See that? There's a TON of 'em.

So, consider that when a site trips one or more of those, Google might THEN send over the transition rank algorithm to be part of a further investigation of the domain.

In that implementation, it becomes a matter of 'trust' - something we know Google is big on. Consider Matt Cutt's most recent post on paid links where he says,

"That’s a clear violation of Google’s quality guidelines, and it’s the reason that [website]‘s PageRank as well as our trust in the website has declined."

Not an uncommon theme. And let's go the other direction, Wikipedia. Certainly a website that makes changes all the time. To the pages, incoming links and so on. What's the difference? A level of domain trust.

it certainly makes a strong reason to leave a penguined/panda site alone and go off and start again on a new site.

This was the contention floated by Tom over at Zdnet. I can guarantee this is not the case. I talked to a Googler yesterday imploring them to chime in on the post, he said that they're tried to rationalize with him in the past, to no avail, so they didn't get involved.

Once more, find the post I just mentioned on web spam/penguin. THOSE are the kind of things that would likely put one in the path of the transition rank algo. Cleaning things up isn't going to qualify a site as spam.

Let's look at ONE of them as an example;

"Keywords in page TITLE: a classic boosting technique and research has found spam pages contain far more keywords than non-spammy (classified) pages"

So... consider I have this page TITLE

Buy blue widgets online

Changing it to;

Buy cheap blue widgets at Wisget Barn

Isn't likely to trip a filter, but this might;

Buy cheap blue widgets online, best blue widgets and widget supplies from the Widget Barm

Then we consider maybe that you also throw some spammy links at it (beyond the thresholds of other listings in the query space) at the same time.

Now you could be in a position where the site could lose some trust. In an attempt to further qualify the site as manipulative, they'd start to have the transition rank approach applied to the site/page.

Obviously I haven't a clue... but this does seem like the more logical approach. I have 7yrs of patent and paper studies and talking to Googlers (in private) which I base my theories on tho. The broad based application of the patent doesn't make as much sense.

So, my current advice is to not get worked up about it. This is one of a few hundred patents awarded each year. We (the industry) need to keep some context by looking at the others as well, not just cherry picking 1-2 per year.

Anyway, there's me 2cc folks. Always up for geeky discussions... keep it goin!

[thegypsy]

Oh and hey, this part made me kinda giggle, and it didn't get much attention during this little drama (out in the blogoshpere)

Unexpected results are bound to elicit a response from a spammer, particularly if their client is upset with the results. In response to negative results, the spammer may remove the changes and, thereby render the long-term impact on the document's rank zero.

This part is REALLY interesting due to the term 'client'. I have NEVER seen this before in web spam patents.

I know for a fact the web spam team knows the difference between a traditional spammer and an SEO. Spammers don't have clients. They build massive networks of site, fire and forget it. They make $5 a month off 1000 sites. Once a site is burned, rinse and repeat.

The fact that this patent talks about 'clients' is a dead give-away that this indeed is about SEOs. That, in the context of this patent, a 'spammer' is most likely an SEO.

A cute little bit that I thought interesting in that more folks didn't really catch it (how many actually read the whole thing...)

To further the point, but not to spread FUD, we can consider some definitions from a Stanford paper

“(...) any deliberate human action that is meant to trigger an unjustifiably favorable relevance or importance for some web page, considering the page's true value.” (from Web Spam Taxonomy, Stanford

and

"Most SEOs claim that spamming is only increasing relevance for queries not related to the topic(s) of the page. At the same time, many SEOs endorse and practice techniques that have an impact on importance scores to achieve what they call "ethical" web page positioning or optimization. Please note that according to our definition, all types of actions intended to boost ranking, without improving the true value of a page, are considered spamming."

Paper is here, and worth reading; [ilpubs.stanford.edu:8090...]

What Matt and company does, is called AIR (adversarial information retrieval). The name alone gives us a hint as to where we stand.

Anyway, just thought to add that tidbit.. later!

[scooterdude]

As a matter of interest, what does Google classify as a keyword ?

Every item in a title is a word, so which ones are defined as key?

Or is it a matter of the judgement of whoever can kick your touche at that time

[Leosghost]

@thegypsy
Ah..so you didn't miss it after all :) I was surprised ..

Agree with almost all of your last two posts..

re:

We (the industry) need to keep some context by looking at the others as well, not just cherry picking 1-2 per year.

This one does strike me as one of the more important ones of the last few years though however, as when added into the mix, it's presence and use confirms so much of what has been visibly going on..if one was paying attention..:)..and not being too narrowly focused in what one was looking at, and the observations conclusions that one could draw from that and ones own research..

[edited by: Leosghost at 2:01 pm (utc) on Aug 31, 2012]

[scooterdude]

thegypsy 2 posts above ring a bell


I'll add that they make a hat from a mould of some ones head, then they see how well the hat fits every one else

If the hat fits you,,,,,

[Leosghost]

How else could algos work ? ..
If it walks like a duck etc ..99% of the time it will be a duck..the occasions when it isn't ..are collateral damage that they are prepared to accept..

What I do take issue with them on though is the damage to SERP rankings of "innocent" sites inherent in the mysterious "shuffle" used to befuddle and identify those who zig zag in response to Google's moves..added to which, even the algorithmic manipulations of SERPS so as to ensnare and ID the SEOs, means that their stated aim of providing the best quality SERPS is not always adhered to..means "the SERPS are the best quality we can give the visitor, except when we are screwing with them for whatever reason we have today"..and there may be very very many "todays"..



But that does fit their "keep it mysterious" and "rule by implied guilt and original sin" concepts that they have adopted from religion..