homepage Welcome to WebmasterWorld Guest from 54.161.220.160
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Obnoxious behavior from 63.86.155.41
Can't track it down...
mivox

WebmasterWorld Senior Member mivox us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 215 posted 8:54 pm on Dec 6, 2000 (gmt 0)

IP 63.86.155.41 made about 200 hits to my site yesterday, in 30 second bursts of 30-50 pages each. It fetched the same pages repeatedly in each visit. I came in to work today, and found the SAME visitor with the same behavior on the stats again...

I can't find ANY info on this IP... the OS is undefined, it doesn't seem to register any concrete browser info, and there's no referrer. A traceroute traced it back to it's own IP... no name on the server.

Here's a clip of one of it's hits from my access logs: 63.86.155.41 - - [06/Dec/2000:07:22:59 -0900] "GET //main.html HTTP/1.1" 200 2848 "-" "Mozilla"

Does anyone have ANY idea who this obnoxious little beast is? Is it someone important, or should I just ban it from my site?

 

mivox

WebmasterWorld Senior Member mivox us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 215 posted 10:43 pm on Dec 6, 2000 (gmt 0)

Well... I hunted some more, found an email address, and got this reply (from the company that controls the IP block containing the offender):

"I have notified the client, Employon.com, and they have stopped this activity. Employon.com is developing spiders which search for companies that post their jobs on the internet, and I'm told that they inadvertently
launched an errant spider, which was operating on a domain database that included your domain. Please let me know if this behavior persists, and I will take additional action.

Digibahn Tech Support"

If you see any super obnoxious 63.X IPs spidering your sites, and reverse IP lookup points to digibahn.com, it's probably the same folks (Employon.com)... But Digibahn is very responsive to the problem!

littleman

WebmasterWorld Senior Member littleman us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 215 posted 2:26 am on Dec 7, 2000 (gmt 0)

Check out this link.
[Employon.com ]
and [beta.grassisgreener.com ]
Looks like they have a whole class C
#########
Employon.com LLC (NETBLK-UU-63-86-155)
22700 Shore Center Drive
Euclid, OH 44123
US

Netname: UU-63-86-155
Netblock: 63.86.155.0 - 63.86.155.255

If they do it again I'd go to grassisgreener.com and right a letter claiming that they are doing a DOS attack. They'll be nice to you after that and exclude your IP out of their crawling.

mivox

WebmasterWorld Senior Member mivox us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 215 posted 2:52 am on Dec 7, 2000 (gmt 0)

Yeah, Employon/Grassisgreener seems to have their very own chunk of UUnet IPs... but a Digibahn email is listed as the "coordinator" under Employon's whois results, so that's who I wrote to.

I sent the same letter to grassisgreener.com... haven't heard back from them. Perhaps the NIC technical contact hadn't the foggiest idea what I was talking about?

Since Digibahn (the ones who seem to be "top dog" over the IP block in question) were so quick to respond, and offered to "take further action" if necessary, I'll probably go to them first if the spider is back tomorrow and just Cc: a copy of the message to grassisgreener.com If Digibahn is in the position to stomp on grassisgreener, they're the ones I want to talk to.

Brett_Tabke

WebmasterWorld Administrator brett_tabke us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 215 posted 10:36 am on Dec 7, 2000 (gmt 0)

Hi mivox - welcome to the world.

That is some very interesting information. A couple months ago, we had a question come up about posting SEO job information here on WebmasterWorld. You just confirmed we were correct in not allowing it to be posted for copyright concerns.

mivox

WebmasterWorld Senior Member mivox us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 215 posted 6:46 pm on Dec 7, 2000 (gmt 0)

Yeah, grassisgreener's spider didn't show much respect for my site... but in all fairness, that particular spider may have still been under development. Their other spiders may observe more respected behavior protocols (asking for a robots.txt, timing page requests more moderately, etc.).

Funny, I always thought that the most obnoxious bots were generally looking for email addresses... now it's job listings.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved