homepage Welcome to WebmasterWorld Guest from 54.211.219.178
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
hostnology.in's domain scraper site
incrediBILL




msg:4383731
 2:26 am on Nov 5, 2011 (gmt 0)

Yup, no shortage of these domain intel scraper sites.

DOMAIN: hostnology.in

IP: 96.30.56.165 (server.avadhwebs.com)

USER AGENT:
"Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

WHOIS of IP:
network:Organization;I:avadhwebs
network:ID:NETBLK-AVADHWEB.96.30.56.165/32
network:IP-Network-Block:96.30.56.165 - 96.30.56.165

Googlebot? That is so lame! LOL!

 

Pfui




msg:4383744
 3:46 am on Nov 5, 2011 (gmt 0)

Bill, the above post and your domain tools OP [webmasterworld.com...] show UAs in quotes.

Are the quotes actually part of the UAs?

If not, could you please not include them? Because some bots -- like SiteIntel [webmasterworld.com...] and the (just-posted) PagePeeker [webmasterworld.com...] -- really do, erm, quote themselves. TIA

incrediBILL




msg:4383751
 5:26 am on Nov 5, 2011 (gmt 0)

sigh

I always post UAs in quotes and use escapes if the quotes are included, like \"

keyplyr




msg:4383782
 10:19 am on Nov 5, 2011 (gmt 0)

I've had that range blocked since last year. My notes say something about a Cugillion SE bot's bad behaviour, also port probe from Cogswell.

96.30.0.0 - 96.30.63.255
96.30.0.0/18

dstiles




msg:4383903
 9:45 pm on Nov 5, 2011 (gmt 0)

The note I have on this range is "wired tree".

incrediBILL




msg:4383941
 11:55 pm on Nov 5, 2011 (gmt 0)

Just realized I only poisoned the page content to catch scrapers and didn't poison the page titles which is all some of these domain intel sites display.

Off to poison my titles now...

update: DONE! checked via proxies, poison titles complete. trap set, now we wait...

dstiles




msg:4384211
 9:58 pm on Nov 6, 2011 (gmt 0)

000h! You are evil. :)

incrediBILL




msg:4384246
 11:51 pm on Nov 6, 2011 (gmt 0)

killer idea moved to new thread: [webmasterworld.com...]

keyplyr




msg:4384262
 12:24 am on Nov 7, 2011 (gmt 0)

poison = killer


I get it, I get it :)

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved