IAS crawler

[keyplyr]

UA: IAS crawler (page scorer; http://integralads.com/site-indexing-policy/)
Protocol: HTTP/1.1
Robots.txt: Yes
Host: AdSafe (integralads.com)
198.148.15.0 - 198.148.15.255
198.148.15.0/24

Integral crawler identifies itself as ia_archiver

Huh?

[keyplyr]

Also...

UA: IAS crawler (ias_crawler; http://integralads.com/site-indexing-policy/)
Protocol: HTTP/1.0
Robots.txt: Yes
Host: AdSafe (integralads.com)
198.148.15.0 - 198.148.15.255
198.148.15.0/24

[lucy24]

Out of curiosity: where did they go after robots.txt? I thought I recognized the name, but it turns out it's because they picked up robots.txt--only--a month or so back, and have never asked for anything else. And it doesn't seem to be because their name overlaps the name of some unwanted robot. Could they be looking for names of specific directories, like something associated with a CMS or ecommerce platform?

[keyplyr]

where did they go after robots.txt?

It may sound odd but I can't remember. That was 2 hours ago and I'm now many miles away from my desk.

[keyplyr]

Just a FYI - I do allow AdSafe, both their range and their bot ias_crawler.