homepage Welcome to WebmasterWorld Guest from 23.20.43.165
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Visit PubCon.com
Home / Forums Index / Marketing and Biz Dev / Cloaking
Forum Library, Charter, Moderator: open

Cloaking Forum

This 129 message thread spans 5 pages: 129 ( [1] 2 3 4 5 > >     
Cloaking Gone Mainstream
Languages, Agents, Doc format, - cloaking is everywhere.
Brett_Tabke




msg:677845
 10:26 pm on Jan 30, 2003 (gmt 0)

Cloaking has taken on so many new meanings and styles over the last few years that we are left scratching our heads as to what cloaking really means. Getting two people to agree on a definition is nearly impossible with all the agent, language, geo targeting, and device specific page generation going on today. It is so prevalent, that it is difficult to find a site in the Alexa top 500 that isn't cloaking in one form or another.

This all came up for us in mid December when right at the height of the Christmas ecommerce season, a friends European site was banned or penalized by a search engine. After numerous inquiries, it was learned that the surprising reason for it was cloaking. I got asked to take a look at the site and figure out where their was a problem. The site owner didn't even know what cloaking was, let alone practice it.

I determined that his off-the-shelf server language and browser content delivery program was classifying search engines as a text browser and delivering them a text version of the page. In it's default configuration, this 5 figure enterprise level package classified anything that wasn't IE, Opera, or Netscape as a text browser and generated a printer friendly version of the page that was pure text.

We explained to the SE just what the situation was and they agreed agreed and took off the penalty after we said we'd figure out a way around the agent part. Unfortunately, the package had all but compiled in the agent support and they were surprised when we informed them about it. What was even better was looking around some fortune 500 companies that run the same software to find three entire sites that were in effect "cloaked" - they didn't have a clue.

In the end we solved the problem with another piece of software that would exchange the agent that the site delivery program was seeing. Yep, we installed cloaking software.

So lets have a little run down of the current state of cloaking in it's various forms:

We've talked a bit about about agent based cloaking recently [webmasterworld.com].

Search Engines Endorse Web Services Cloaking:

Cloaking has become just varying shades of gray. We now have instances where search engines themselves endorse cloaking (xml feeds) and in some instances are giving out cloaking software to deliver those xml feeds.

That has resulted in pages intended (cloaked) for one search engine being indexed by another search engine. There have been occasions where this endorsed content has been banned or penalized by another search engine.

Geographic IP Delivery:

Language translations have been a hot topic for the last year. Most major sites now geographic deliver content in one form or another. Hardly a month goes by when someone doesn't scream, I can't get to Google.com because they are transparently redirected to a local tld. You will also find those same search engines custom tailoring results for that IP address (eg: personalized content generation). You can see the effect your self by changing your language preferences on a few search engines that offer the feature.

One Browser Web:

The recent history of major browsers is summed up in IE4-6, and Netscape 3-7. There is also a large 2nd tier of browsers: Opera, Lynx, Icab, and Mozilla.

All of these agents support different levels of code and standards. They also have inherent bugs related to page display. If you are a web designer, you could get a degree in the various browser differences of CSS and HTML alone.

Just when we are starting to think in terms a one browser web, along comes a whole new set of browsers to consider: Set Top Boxes, Cell Phones, PDA's, and other Mobile Devices. These all have varying degrees of support for XML, XHTML, CSS2/3, and the web services protocol blizzard (eg: .net, soap...etal).

We've not even begun to talk about IE7 which is rumored to be in final internal beta testing. Then there is Apples new browser and the growing horde of Mozilla based clones. When you put it in those terms, our one browser web seems like a distant dream.

Delivering different content to these devices is a mission critical operation on many sites. Generating content for mobile devices is a vastly different proposition than delivering an xml feed to a search engine, or a css tricked out page for a leading edge browser.

Given that the combination of vistor ip and user agent can run into hundreds of possibilities, the only valid response is agent and ip cloaking.

Off the shelf cloaking goes mainstream.

There many off-the-shelf packages available today that include cloaking in one form or another. The perplexing part is that many sites are cloake in ways you wouldn't even know about. There are several major forum packages that cloak in some form or another.

I was at a forum this morning that was agent cloaking, and other that was language cloaking. In both cases, the webmasters don't even know that it is taking place - let alone have the tech knowledge to correct it.

Welcome to 2003 - Modern Era Of Search Engines.

This isn't the web of 98-99 where people would routinely get whisked away to some irrelevant site unrelated to their query. Todays search engines are vastly improved with most engine algorithms putting Q&A tests on every page they include. Those range from directory inclusion requirements, inbound link count and quality, to contextual sensitivity and even a pages reputation.

In this modern era where search engines now routinely talk about their latest off-the-page criteria algo advancements, it's clear that traditional se cloaking has little effect. It comes down to one simple fact, those that complain about SE cloaking are simply over looking how search engines work. The search engines have done a fantastic job at cleaning up their results programatically and by hand.

The most most fascinating thing about this new main stream cloaking is the situation where a site just classifies a search engine as a graphically challenged browser. In that case, cloaking becomes mostly a agent based proposition. The trouble starts when you throw in language delivery to the equation, or even delivering specific content as part of a search engine program.

All of these wide ranging factors combined to result in about 10 to the 4th power of page generation possibilities. In that situation, it almost becomes a necessity to put spiders into the all text browser category and deliver the same page to the se's that you deliver to cell phones or the Lynx browser.

Thus, we've come full circle on search engine cloaking. We no longer cloak to deliver custom content to search engines, we now cloak for the search engines to keep them from getting at our cloaked content for visitors.

<edit> cleaned up some typos and syntax errors</edit>

[edited by: Brett_Tabke at 6:15 am (utc) on Feb. 3, 2003]

 

rogerd




msg:677846
 10:38 pm on Jan 30, 2003 (gmt 0)

Great story, Brett, and certainly an amusing solution to the problem. Like many gray areas, cloaking is clearly being adopted by so many mainstream sites that the old SE approach of comparing what its robot gets to a bot that looks human gets is outmoded.

I'm not sure where this will lead... More cloaking for SEO purposes? SE development of "cloaking comparison" algorithms to rate the probability that a site is cloaking for spam purposes?

Certainly the days of a search engine declaring, "if we catch you cloaking, you are out" are gone (or nearly so) - a human review would be all but mandatory for this kind of decision now.

korkus2000




msg:677847
 10:54 pm on Jan 30, 2003 (gmt 0)

What about technologies that are new like Flash that SEs can't see. I think cloaking can help SEs where they are lacking in some instances.

webwhiz




msg:677848
 4:59 am on Jan 31, 2003 (gmt 0)

Sorry, but I must disagree that the things Brett is talking about are cloaking. That's much too broad of a definition, and only serves to muddy the waters as to what cloaking is and what it isn't, and whether it's something you can use as an SEO technique or not.

One doesn't need to find out the search engine spider IP to do those things, and therefore the search engines have no problem with them. They're not cloaking at all. They are simply what they are, i.e., Geographic IP Delivery, etc.

Marcia




msg:677849
 6:24 am on Jan 31, 2003 (gmt 0)

Thanks for stopping by Jill, I always enjoy your posts.

Sorry, but I must disagree that the things Brett is talking about are cloaking.

Would you be kind enough to point out the specific points for us that you're referring to? I know you're a whiz with words; perhaps if you could redefine and clarify what certain things might better be called, we'd all gain more insight and become more semantically correct.

I'm plain vanilla myself, have never gone near cloaking or anything like it. I'm afraid it's far too techie for a simple gal like myself, but I always enjoy reading about it.

Woz




msg:677850
 7:07 am on Jan 31, 2003 (gmt 0)

>what cloaking is and what it isn't

Cloaking is a tool just the same as any other tool available to webmasters in their activities of creating and promoting websites. It is the use of those tools, including, but certainly not limited to, cloaking, and varying opinions on what constitutes acceptable or not-acceptable usage thereof that muddys the waters.

The waters will never be completely clear as with any contentious issue. We as individuals or employees simply have to chose what level of clarity is acceptable to us, or deemed acceptable by employers, as we persue our goals, and allow others to choose different currents in the waters based on their own beliefs and directions.

To each his own.

Onya
Woz

jeremy goodrich




msg:677851
 7:09 am on Jan 31, 2003 (gmt 0)

Hi Jill -> good to see you 'popped in' again.

Um, ya -> I'd like a bit more 'clarification' too. As somebody that's been cloaking, more or less, for a good three years and been getting rankings in nice places for it :) perhaps you can tell me what I was missing when I beat the other chaps at getting to the top?

Oh, and while you're at it, we have a great site search [searchengineworld.com] I mentioned a few months ago that MSN (the UK one) was cloaking their home page. :)

If you don't mind, please fill us all in on why that was wrong...or, as Marcia mentioned, you could just fill us in on the specifics of what Brett said that was unclear.

Don't worry, I'll understand the technical parts too. I'm sure most of the members will, as well.

georgeek




msg:677852
 8:29 am on Jan 31, 2003 (gmt 0)

Brett's excellent summery of cloaking reinforces the view that the term itself is undergoing semantic stress and some new words are now needed to regain precision. Directing search engines to index what the public cannot see for the purpose of achieving high rankings is in a sense fraudulent.

'Fraudulent cloaking' is a bit of a mouthful how about 'floaking'?

Nick_W




msg:677853
 8:35 am on Jan 31, 2003 (gmt 0)

Fraudulent againt whom?

If I have a heavy flash page that contains alot of information on my widgets site, where is the problem with making sure an SE can access that data also? - Not to mention visually impaired users and people that can't abide flash ;)

The problem is it's not black and white and saying that all cloaking is evil is kinda silly (not that you did say that georgeek :)), it misses the point in it's genarlism and just serves to confuse the issue further.

I too would welcome some clarification from webwhizz, I'm afraid I really didn't understand what you were trying to say?

Nick

georgeek




msg:677854
 9:07 am on Jan 31, 2003 (gmt 0)

Fraudulent againt whom?

Those that are deceived.

If I have a heavy flash page that contains alot of information on my widgets site, where is the problem with making sure an SE can access that data also?

I think you have to make it clear what you mean by that data. In your example if that data is simply a translation of flash to something more digestible - quite clearly it is not fraudulent. If that data is constructed for the purposes of deception - then it is.

In this thread I am wearing my devil's advocate hat :)

webwhiz




msg:677855
 12:52 pm on Jan 31, 2003 (gmt 0)

Well, in order to best explain what I meant, it would be easiest to simply point you to the article that Alan Perkins wrote for my newsletter this week. I know you guys aren't big on links, but hopefully the moderator will indulge me. If not, go to the Issue 041 in my archives:
[highrankings.com...]

The gist of it is that the word cloaking is being used for applications that shouldn't really be called cloaking (in Alan's opinion and also in mine). By calling perfectly legitimate things that the engines have no problem with, such as Geographic IP Delivery, it legitimizes the "real" type of cloaking, i.e., the kind where you need to find out the SE bot's IP address so you can feed it something strictly designed for high rankings. It's all about tricking the search engines, as opposed to helping the users.

Those other applications are fine. The engines have no problem with them, and they are great for users too. But they're not cloaking, as cloaking is something you do to trick, not help.

Alan said it a lot more eloquently and logically than I did, so I hope you will read the article and then I'd love to hear your comments!

msgraph




msg:677856
 1:09 pm on Jan 31, 2003 (gmt 0)

ok.

I never even heard of Alan Perkins before.......

lawman




msg:677857
 1:30 pm on Jan 31, 2003 (gmt 0)

I never even heard of Alan Perkins before.......

You know, Wild Kingdom. ;)

webwhiz




msg:677858
 1:55 pm on Jan 31, 2003 (gmt 0)

You know, Wild Kingdom.

:D Well, he really went into the lions den (again) this time!

korkus2000




msg:677859
 2:09 pm on Jan 31, 2003 (gmt 0)

Google Cloaking Policy [google.com]
The term "cloaking" is used to describe a website that returns altered webpages to search engines crawling the site. In other words, the webserver is programmed to return different content to Google than it returns to regular users, usually in an attempt to distort search engine rankings. This can mislead users about what they'll find when they click on a search result. To preserve the accuracy and quality of our search results, Google may permanently ban from our index any sites or site authors that engage in cloaking to distort their search rankings.

So by this definition serving up a googlebot readable page for a Flash site is "cloaking". I don't understand a problem with overcoming googlebot's accessibility problem with Flash. When Google actually implements Macromedia Flash Search Engine SDK [macromedia.com] this won't be an issue. AllTheWeb has already implemented it for 4 months [webmasterworld.com].

Just as Woz says above, "cloaking" is a tool like Flash. It can be used by the dark or light side. Like Flash [webmasterworld.com] there are people who have seen to many bad implementations that all usage should stop.

I am not an SEO. I am a web designer. I have no clients to get high rankings for. I just make web sites. "Cloaking", the way it applies to my industry, is a way to use current technology, instead of waiting decades for the search engines to catch up. How old is Flash? When will Flash be indexable? Why should something that has a 90% penetration rate [webmasterworld.com] on the Internet be blacklisted by the search engines? I wish someone could explain this to me.

4eyes




msg:677860
 2:22 pm on Jan 31, 2003 (gmt 0)

..the word cloaking is being used for applications that shouldn't really be called cloaking (in Alan's opinion and also in mine)

That is the crux of the problem.

Clearly one word is no longer enough for this sensitive and misunderstood topic.

Any discussion on the ethics of 'cloaking' just turns into a battle of different definitions.

Personally I find it hard to see anything other than shades of grey here.

Even organising an orchestrated link popularity campaign has an element of manipulation in it, and there are many well known names on both sides of this argument that are actively engaged in link chasing.

There is no moral high ground - there is just 'high risk' and 'low risk'.

webwhiz




msg:677861
 2:24 pm on Jan 31, 2003 (gmt 0)

So by this definition serving up a googlebot readable page for a Flash site is "cloaking".

Huh? Absolutely not! When you serve up a readable page for the search engines (as opposed to the flash one) you don't need the search engine bot's IP to do so! Where did you get that idea?

Your comment is exactly the thing Alan was trying to clear up in the article. Apparently it's still not understandable to some?

korkus2000




msg:677862
 2:31 pm on Jan 31, 2003 (gmt 0)

Your comment is exactly the thing Alan was trying to clear up in the article.

I am not talking about Alan's article. I am using Google's definition. I am not writing a rebuttal to an article. I am adding to the thread about the nice shades of gray that the term "Cloaking" has. I didn't read in Google's definition anything about IPs.

rogerd




msg:677863
 2:41 pm on Jan 31, 2003 (gmt 0)

I think it's really hard draw a firm line. The "I know cloaked spam when I see it" argument doesn't really hold water when it's not blatant.

Here's an example: just about everyone in this thread seems to agree that feeding text content to user agents that can't handle Flash is OK, or at least should be OK. We also know that hidden text is frowned upon, and unless you are Jakob Nielson, may get you penalized. Well, if I'm feeding text to non-graphic user agents, I'm going to be sure it contains my keywords. The only difference between my text page and hidden text is I won't have to make it match the background color and it may have to sound like proper English.

How would you like to be the SE tech trying to figure out if the text content is a good substitute for the graphic presentation, or if it crosses the line? Sure, you can catch the blatant spammers and keyword packers, but a well-written and formatted page is going to be a different story (even if it is keyword-laden). This is a tough one for the SEs - if they acknowledge that cloaking (or "providing alternative content") is OK for some purposes, it really opens Pandora's box.

Marcia




msg:677864
 2:45 pm on Jan 31, 2003 (gmt 0)

Your comment is exactly the thing Alan was trying to clear up in the article. Apparently it's still not understandable to some?

See now Jill, that's the problem with linking to articles, and is the very reason why we don't allow it, preferring to keep the discussions here at the board where there's a dynamic, interactive interchange of ideas.

That way everyone understands instead of it being shrouded in mystery. Let's go back to the original post in this thread, which technically, according to our TOS and regular posting protocols, is what we're discussing.

We'd prefer to hear what you have to say. If we're not understanding the points in Brett's post that you're disagreeing with, why don't you clear it up for us and explain it to us in your own words.

webwhiz




msg:677865
 2:48 pm on Jan 31, 2003 (gmt 0)

roger, what you're talking about isn't cloaking. It might be considered spam, depending on what you were actually putting on the page for the non-graphical browsers, but it's not cloaking if you don't need to first know the search engine bot's IP address to feed them the info.

It's really pretty simple when you use that as your definition. Unfortunately, not everyone is prepared to use that as a definition. It would clear up so much, however, and there would not be those gray areas.

Again, I'm talking about cloaking, not spam. Spam will always have gray areas. Cloaking doesn't have to at all under that definition.

webwhiz




msg:677866
 2:51 pm on Jan 31, 2003 (gmt 0)

We'd prefer to hear what you have to say. If we're not understanding the points in Brett's post that you're disagreeing with, why don't you clear it up for us and explain it to us in your own words.

I kinda thought I did that here:
The gist of it is that the word cloaking is being used for applications that shouldn't really be called cloaking (in Alan's opinion and also in mine). By calling perfectly legitimate things that the engines have no problem with, such as Geographic IP Delivery, it legitimizes the "real" type of cloaking, i.e., the kind where you need to find out the SE bot's IP address so you can feed it something strictly designed for high rankings. It's all about tricking the search engines, as opposed to helping the users.

Those other applications are fine. The engines have no problem with them, and they are great for users too. But they're not cloaking, as cloaking is something you do to trick, not help.


Laisha




msg:677867
 3:39 pm on Jan 31, 2003 (gmt 0)

I never even heard of Alan Perkins before.......

Sound designer for the 1998 blockbuster "Zacharia Farted." [entertainment.msn.com]

DaveAtIFG




msg:677868
 4:38 pm on Jan 31, 2003 (gmt 0)

the kind where you need to find out the SE bot's IP address so you can feed it something strictly designed for high rankings
So user_agent cloaking is still acceptable to you? I'm relieved! ;)

It's all about tricking the search engines, as opposed to helping the users.
There's a thread here [webmasterworld.com] proposing the SEs offer an "uncloaking service," and a few responses. Reiterating one of my comments in that thread:
If cloaking represented a major source of irrelavent results in the SERPs, it would affect the SEs income and "bottom line" and they would have discouraged it years ago instead of doing lip service in their webmaster dos and don'ts pages.

webwhiz




msg:677869
 4:50 pm on Jan 31, 2003 (gmt 0)

So user_agent cloaking is still acceptable to you? I'm relieved!

It would be user agent delivery, I believe, not cloaking. Do you need the search engine's IP to do it? If not, then yep, perfectly acceptable. Doesn't matter what's acceptable to me though, it's what's acceptable to the search engines that matter.

I don't believe the search engines have a problem with it either. It's not against any rules or guidelines as far as I know.

korkus2000




msg:677870
 4:57 pm on Jan 31, 2003 (gmt 0)

Why is the IP so important to you? People can set up user agents to say they are anything. The IP just makes sure you are targeting what you want to target. What is the difference between IP and User agent cloaking? Where does it say that google doesn't like IP cloaking but does like user agent cloaking?

Nick_W




msg:677871
 5:00 pm on Jan 31, 2003 (gmt 0)

Good points Korkus. This argument seems full of flaws and misunderstandings to me.

And besides, I thought it was only cheating if you got caught? ;)

Nick

DaveAtIFG




msg:677872
 5:12 pm on Jan 31, 2003 (gmt 0)

It's not against any rules or guidelines as far as I know.
From relieved to confused... :(

The Google guidelines [google.com] state in part:
The term "cloaking" is used to describe a website that returns altered webpages to search engines crawling the site. In other words, the webserver is programmed to return different content to Google than it returns to regular users
Whether it's UA or IP based, people cloak to provide search engine friendly pages to spiders... Appears to me as if your (or Alan's) definition of cloaking may need some polish.

webwhiz




msg:677873
 5:25 pm on Jan 31, 2003 (gmt 0)

Whether it's UA or IP based, people cloak to provide search engine friendly pages to spiders... Appears to me as if your (or Alan's) definition of cloaking may need some polish.

I don't believe the definition needs any polish. Other users besides the search engines can and do see the UA content you're talking about. i.e., handhelds, other browsers, etc.

Having to need the search engines IP means only one thing...trying to trick the search engines. Nobody else EVER sees that content that you feed the engine. There's no other purpose for it. And that is the ultimate definition of cloaking.

The other stuff isn't cloaking at all. Sometimes it can be used to spam, but it's not cloaking.

seth_wilde




msg:677874
 5:34 pm on Jan 31, 2003 (gmt 0)

Do you need the search engine's IP to do it? If not, then yep, perfectly acceptable

I'm going to have to agree with the others that this is a flawed logic. I don't understand how you can say that I can send googlebot one page, scooter a different page, and then send IE yet a different page and that's ok, but if I do essentially the same thing by using IP's it's bad.

It's still cloaking, it's just sloppy cloaking..

This 129 message thread spans 5 pages: 129 ( [1] 2 3 4 5 > >
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Marketing and Biz Dev / Cloaking
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved