This Page Is Cloaked

[startup]

If I hadn't told you that this page was cloaked you wouldn't have known.
Now let's pretend that this is one of your competitors. They may be cloaking, and your job is to find out if they are.

This is an excercise to see if you can get to the search engine page that your would-be competitors are using.

At this time the script is set to the most secure form of cloaking, IP delivery.
-May 16 I will change the settings to a combination of IP delivery, User Agent and spider detection. At this point, some of you may get the script to deliver the cloaked page.
-Each day for the rest of the week I will "relax" the setting on the script, until it should be fairly easy for most of you to get to the cloaked page.

Have Fun :).

This is not an attempt at any form of promotion. If anyone chooses to engage in promoting any script and/or service in any responses to this posting, I would like the moderator, (Air), administrator, (Littleman), owner, (Brett) to delete the whole thread.

Oh did I mention, have fun :).
[auctionlsoftware.hypermart.net...]

[Brett_Tabke]

Does calling in a favor from a search engine count? lol

Used the standard methods and it appears tight.

[startup]

Ah come on Brett, if that page was #1 for "farm equipment" you would have used alot more than the
regular bag of tricks to bust through:).

The only time I would use an SE favour is on Mother's Day;).

[Brett_Tabke]

>more than the regular bag of tricks to bust through

What, and expose the _good stuff_? never. hehe

[grnidone]

Got it.
-G

[startup]

"Got it", sneaky.
I have just relaxed the settings on the script.

Let's see who can get to the cloaked pages now.

[jeremy goodrich]

Wait a second. I've already faked 2 different user agent strings, and I still didn't get it.

I thought I only need the HTTP_USER_AGENT variable now? Or am I mistaken? Or since the contest is already going, I don't get any more info?

[4eyes]

Startup - once the game is over will you let us know how many tries were made and how many got through.

great fun

[startup]

Jeremy,
I "relaxed" the settings on the script to the point that a "real spider" using a valid User Agent" would get through. There are many spiders available to you on the web that will spider a page but not meet the criteria of a "Se spider".

I have just "relaxed" the script setting again, now the basic spider and matching User Agent should get you to the cloaked page.

At 5:00pm EDT the setting will be changed again and the User Agent spoofing should work.

4eyes,
Now that sounds like a great idea. The whole idea was to have fun. Once you knew the page was cloaked, the fun was how to get to the real page.

[bobriggs]

Ok I 'cracked' the google version, using:
Googlebot/2.1 (+http://www.googlebot.com/bot.html)

But when I send this header:

Accept-Language: en

I only get the regular visitor version.

Why?

[Air]

Grnidone,

>Got it.

you never did say how you got it, or which page, it might be interesting for the folks following this thread if you want to give more info.

[Q]

Finally got it with a Northern Light UA. Had used all the usual "tricks" the past couple days.

I would love to know how you got it grnidone.

[startup]

The page is now cloaked only using User Agent.

"Got it",
I was very busy when you posted, now that I have had time to check the log files, I have yet to figure out what it is you got.
Were you responding to Brett's reply?

bobriggs,
I am still going through the log files.

[bobriggs]

Whoops. Did you relax any more restrictions since my post?

I can send an accept-language header now and get the results.

<h2>Congratulations!! You have cracked the <font color="#FF0000">Google</font> version of this page</h2>
<h2>Congratulations!! You have cracked the <font color="#FF0000">Fast/Alltheweb</font> version of this page</h2>

One thing that I did try though was using a newer spider on FAST:

FAST-WebCrawler/2.2.5 ([e-mail]; [fast.no ]).

I had been seeing
FAST-WebCrawler/2.2-pre45 ([e-mail]; [fast.no ])
until today.

[startup]

Yes I relaxed the setting at 5:00pm EDT

[startup]

Total visits=206
spiders =36
"Sim Spiders"=57

The first to get through:

5/18/01 8:43am EDT,
User Agent: Googlebot/2.1 (+http://www.googlebot.com/bot.html)
IP Address: 172.170.159.92. The cloaked page was delivered based on the User Agent and the spider that was used. At this point, as long as a valid User Agent and spider was used, the cloaked page was delivered.

May 15/16, the script was using IP address and matching IP to User Agents.
May 17, User Agent and spider detection was being used. Please note that I have yet to see anything other than an SE spider satisfy the spider criteria.
May 18, before 5:00pm EDT. A basic spider and User Agent would get the cloaked page delivered.

May 18, after 5:00pm. The cloaking is based only on User Agent.

The real winner is bobriggs, getting to the cloaked page in as many different ways as possible.

I hope everyone enjoyed themselves and now has a new opinion of cloaking.

[seostud]

Is there anyway that I tell that certain rankings use a cloaked page or not or is this impossible?

[4eyes]

startup

well done - great exercise and great game.

thanks

[startup]

Thanks, I had alot of fun doing it.
Cloaking has many legitimate uses and I really believe it has got a bad name from the abuse. Next time you see, "index.cgi or index.asp" think about the script that delievered the page.

[pathos]

Neat game! I wonder if there's any way to do a "search engine challenge" every few weeks? I got here a little to late to play this time, but maybe next time...