Shopwiki bot IPs

[dstiles]

Does anyone know if shopwiki bot uses IPs in the ranges below? (It's so annoying not being able to specify the exact IP!)

65.46.75.nnn
38.98.120.nn
38.98.120.nn
67.152.29.nnn

I've had them allocated for a couple of months now but when I found the bot was using the full 209.51.162.0 - 209.51.162.255 range with proper rDNS I dumped the others, which don't have a bot-type rDNS, in the belief they had been discontinued.

The 67.152.29.13n one from the XO range came round just now and was blocked but, unexpectedly, it occasionally had the Referer set to the home URL (http://www.example/com/) which struck me as odd for a bot. I haven't seen Referers present from the new range so I wondered if it's just a very crafty scraper.

[keyplyr]

Just a FYI, but they have always obeyed robots.txt on my sites:

User-agent: ShopWiki
Disallow: /

[tangor]

They have also honored my general disallow since I whitelist a handful and exclude all the rest.

[dstiles]

Seems fine here across several sites, some allowed in robots.txt and others disallowed.

My problem is: I'm not sure if the IPs I listed are really shopwiki or impostors, given no rDNS.

I forgot to add a couple of other IPs that seem to be genuine: 8.11.2.9 - 8.11.2.10 - in fact the whole /24 seems to be them but those are the only ones that I've seen crawl. rDNS for the pair is not bot-like, though.

[Pfui]

I've not had the positive experiences y'all report:

ShopWiki
Really messed up; really messes things up.
[webmasterworld.com...]

[dstiles]

Sorry, didn't notice that thread when I posted. :)