- - Marketing and Biz Dev
- -- Cloaking
- ---- How exactly do search engines detect cloaking?
volatilegx - 1:49 pm on Aug 2, 2007 (gmt 0)
|It's technically impossible without cheating. |
There aren't really any rules to the game, other than what the individual webmasters set, so how can they be cheating?
Webmasters can set rules by denying access to anybody they want, cloaking certain content, requiring user registration, etc.
Back to the main discussion, search engines try to discover cloakers by using a number of techniques:
- By visiting using an IP address which is not registered to their company.
- By visiting using a non-spider user agent
- By comparing caches from different sources that they own; i.e., from their main spider and from their page accelerator
- By comparing caches from different sources that they don't own; i.e., from their main spider and from some other company's cache
- By using an algorithmic process to identify likely candidates and using human editors to verify.
|Also, while I'm asking, why didn't HTTP include an "Is-robot" header? |
It might be nice, but it would also be really easy to abuse. Why use the header... what benefit would it bring the owners of bots?
Thread source: http://www.webmasterworld.com/cloaking/3406490.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com