homepage Welcome to WebmasterWorld Guest from 54.204.73.126
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Defense contractor can't set User_Agent
littleman

WebmasterWorld Senior Member littleman us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 691 posted 9:45 pm on May 21, 2001 (gmt 0)

ip -> 204.7.154.4
Host name -> athens.psrw.com
U_A -> $agentname ;)

This is their very rude product.
[psrw.com...]

 

mivox

WebmasterWorld Senior Member mivox us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 691 posted 9:49 pm on May 21, 2001 (gmt 0)

Does it ask for robots.txt? It would make me laugh to add an entry saying
User-agent: $agentname
Disallow: /

I wonder if a robots.txt line like that would confuse other spiders? Perhaps safer to just ban it by IP?

Air

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 691 posted 10:40 pm on May 21, 2001 (gmt 0)

That's funny LM, I wonder what the name intended for it really was, or did they mess up and use two $$ signs on their scalar assignment and ended up naming the scalar instead of assigning a value --> probably not important why I guess ...

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved