| Welcome to WebmasterWorld Guest from 188.8.131.52 |
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
|Pubcon Platinum Sponsor 2014|
|WebmasterWorld Uses Cloaking!|
User agent specific delivery for spiders
| 9:33 am on Jan 2, 2009 (gmt 0)|
For those of you that weren't aware, WebmasterWorld itself has a cloaked robots.txt file that only shows the real robots.txt file to Yahoo Slurp, MS's msnbot, Ask Jeeves or Google's googlebot.
Here's the PERL code:
It's a very simple example of how you can send one file to the spiders or a different file to everyone else.
| 3:19 pm on Jan 5, 2009 (gmt 0)|
| 4:38 pm on Jan 5, 2009 (gmt 0)|
Ya, when it was updated. I thought the concept was brilliant when Brett first launched it. The dust has accumulated...
OMG! WebmasterWorld cloaks? ;)
| 5:05 am on Feb 3, 2009 (gmt 0)|
Good one, incrediBILL
Just why does the copyright notice need to be posted twice, header & footer ?
| 6:02 am on Feb 3, 2009 (gmt 0)|
> ... why does the copyright notice need to be posted twice ... ?
It was stipulated by the Head Chief Lead Tautologist of our Department of Redundancy Department.
But seriously, probably so that those inclined to forget quickly would be reminded again at the end of their scrolling down through the page. Not a bad idea actually...
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved