Domain Tools Now Using GurujiBot

IP is from Name Intelligence

[incrediBILL]

I give all the sites I block feedback pages so I can identify who they are when I find that feedback data being used and this one came as a shocker.

On the Domain Tools page they have an SEO Text Browser claims that it crawled from their IP range (Name Intelligence) but the user agent has changed to Gurujibot!

66.249.16.212
Mozilla/5.0 (compatible; GurujiBot/1.0; +http://www.guruji.com/en/WebmasterFAQ.html)

Name Intelligence, Inc. NAMEI-NET-1 (NET-66-249-16-0-1)
66.249.16.0 - 66.249.17.255

Not sure if Domain Tools is helping GurujiBot crawl in exchange for the plain text cache of the page or what, but this is certainly an interesting change.

[edited by: incrediBILL at 2:09 am (utc) on Sep. 13, 2009]

[incrediBILL]

Quick check in Guruji itself and my pages were (attempting) to be indexed from here:

72.20.109.62 = guruji.com.
Mozilla/5.0 GurujiBot/1.0 (+http://www.guruji.com/en/WebmasterFAQ.html)

Bay Area Internet Solutions BAYAREA-BLK-1 (NET-72-20-96-0-1)
72.20.96.0 - 72.20.127.255

Guruji.com BAYA-72-20-109-32 (NET-72-20-109-32-1)
72.20.109.32 - 72.20.109.63

So it had no relation to what I saw on Domain Tools, very odd.

[Pfui]

I've blocked Bellevue, WA-based Name(no-space)Intelligence for a long time because of NI's stealth behavior, its robots.txt-related log-spamming and its Domain(no-space)Tools-related thumb-nailing.

That said, if you ever see DT referers, you're most likely seeing the hosts and UAs of people/bots checking out the info DT has about your site as displayed on one of its results pages.

(Alternatively, since their WHOIS info shows up in Googles's SERPS when you use a domain as a keyword, someone could be doing a large-scale somethingorother with your URL through Google.)

You can see the Host+DT referer in action when you check any of your sites. Go to DT's 'page' for your site, then click on the thumbnail. Then check your logs. just You should see the host+UA you used, and a DT referer in this format:

[whois....] [DT] .com/yoursitenamehere.com

Then again, if the above isn't related to your OP... "Nevermind." :)

Note: NI and DT used above because WW's program obfuscates their no-spaced names.

[Pfui]

Correction: The UA is the viewer's; the Host/IP is Spry's (server farm; NI host):

66.249.16.nnn
Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; en-us) AppleWebKit/531.9 (KHTML, like Gecko) Version/4.0.3 Safari/531.9