homepage Welcome to WebmasterWorld Guest from 54.211.219.68
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
elefent/Elefent 1.1
keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4515424 posted 6:59 pm on Nov 3, 2012 (gmt 0)


elefent/Elefent 1.1 (Friendly web walker.; http://elefent.eu/; jason@elefent.eu)
robots.txt: yes

Looks like it's running from a private ADSL account on CZ.CZNET (NEXTEL-XDSL)
90.177.160.0 - 90.177.160.255

 

incrediBILL

WebmasterWorld Administrator incredibill us a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month



 
Msg#: 4515424 posted 9:49 pm on Nov 3, 2012 (gmt 0)

I don't let things crawl that give no more information than "Coming Soon" on their website.

I saw one very aggressive crawler post such a site for many years and nothing came from their scraping that we know about.

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4515424 posted 10:25 pm on Nov 3, 2012 (gmt 0)

Friendly web walker


some new words to add into the NO-NO list ;)

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4515424 posted 11:45 pm on Nov 3, 2012 (gmt 0)

Is it just me, or is the .eu by itself enough to raise suspicion? (I tend to confuse it with .su, to which the answer would defitely be "It's not just you" ;) ) What humans browse from .eu domains?

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved