Welcome to WebmasterWorld Guest from 54.196.214.35

Forum Moderators: open

Message Too Old, No Replies

WebmasterWorld Uses Cloaking!

User agent specific delivery for spiders

     
9:33 am on Jan 2, 2009 (gmt 0)

Administrator from US 

WebmasterWorld Administrator incredibill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Jan 25, 2005
posts:14624
votes: 88


For those of you that weren't aware, WebmasterWorld itself has a cloaked robots.txt file that only shows the real robots.txt file to Yahoo Slurp, MS's msnbot, Ask Jeeves or Google's googlebot.

Here's the PERL code:
[webmasterworld.com...]

It's a very simple example of how you can send one file to the spiders or a different file to everyone else.

3:19 pm on Jan 5, 2009 (gmt 0)

Senior Member

WebmasterWorld Senior Member quadrille is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Feb 22, 2002
posts:3455
votes: 0


Great blog!
4:38 pm on Jan 5, 2009 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member pageoneresults is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 27, 2001
posts: 12166
votes: 51


Great blog!

Ya, when it was updated. I thought the concept was brilliant when Brett first launched it. The dust has accumulated...

OMG! WebmasterWorld cloaks? ;)

5:05 am on Feb 3, 2009 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Nov 26, 2002
posts:810
votes: 0


Good one, incrediBILL

Just why does the copyright notice need to be posted twice, header & footer ?

6:02 am on Feb 3, 2009 (gmt 0)

Senior Member

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Mar 31, 2002
posts:25430
votes: 0


> ... why does the copyright notice need to be posted twice ... ?

It was stipulated by the Head Chief Lead Tautologist of our Department of Redundancy Department.

But seriously, probably so that those inclined to forget quickly would be reminded again at the end of their scrolling down through the page. Not a bad idea actually...

Jim