|Obnoxious behavior from 18.104.22.168|
Can't track it down...
IP 22.214.171.124 made about 200 hits to my site yesterday, in 30 second bursts of 30-50 pages each. It fetched the same pages repeatedly in each visit. I came in to work today, and found the SAME visitor with the same behavior on the stats again...
I can't find ANY info on this IP... the OS is undefined, it doesn't seem to register any concrete browser info, and there's no referrer. A traceroute traced it back to it's own IP... no name on the server.
Here's a clip of one of it's hits from my access logs: 126.96.36.199 - - [06/Dec/2000:07:22:59 -0900] "GET //main.html HTTP/1.1" 200 2848 "-" "Mozilla"
Does anyone have ANY idea who this obnoxious little beast is? Is it someone important, or should I just ban it from my site?
Well... I hunted some more, found an email address, and got this reply (from the company that controls the IP block containing the offender):
"I have notified the client, Employon.com, and they have stopped this activity. Employon.com is developing spiders which search for companies that post their jobs on the internet, and I'm told that they inadvertently
launched an errant spider, which was operating on a domain database that included your domain. Please let me know if this behavior persists, and I will take additional action.
Digibahn Tech Support"
If you see any super obnoxious 63.X IPs spidering your sites, and reverse IP lookup points to digibahn.com, it's probably the same folks (Employon.com)... But Digibahn is very responsive to the problem!
Check out this link.
and [beta.grassisgreener.com ]
Looks like they have a whole class C
Employon.com LLC (NETBLK-UU-63-86-155)
22700 Shore Center Drive
Euclid, OH 44123
Netblock: 188.8.131.52 - 184.108.40.206
If they do it again I'd go to grassisgreener.com and right a letter claiming that they are doing a DOS attack. They'll be nice to you after that and exclude your IP out of their crawling.
Yeah, Employon/Grassisgreener seems to have their very own chunk of UUnet IPs... but a Digibahn email is listed as the "coordinator" under Employon's whois results, so that's who I wrote to.
I sent the same letter to grassisgreener.com... haven't heard back from them. Perhaps the NIC technical contact hadn't the foggiest idea what I was talking about?
Since Digibahn (the ones who seem to be "top dog" over the IP block in question) were so quick to respond, and offered to "take further action" if necessary, I'll probably go to them first if the spider is back tomorrow and just Cc: a copy of the message to grassisgreener.com If Digibahn is in the position to stomp on grassisgreener, they're the ones I want to talk to.
Hi mivox - welcome to the world.
That is some very interesting information. A couple months ago, we had a question come up about posting SEO job information here on WebmasterWorld. You just confirmed we were correct in not allowing it to be posted for copyright concerns.
Yeah, grassisgreener's spider didn't show much respect for my site... but in all fairness, that particular spider may have still been under development. Their other spiders may observe more respected behavior protocols (asking for a robots.txt, timing page requests more moderately, etc.).
Funny, I always thought that the most obnoxious bots were generally looking for email addresses... now it's job listings.