Procter & Gamble bot-running

(not just soap 'n' stuff)

[Pfui]

Hey, all. LTNS. FYI: For a few weeks now, I've seen variations of the following CIDR's IPs sniffing around once a day or so:

IP: 143.20.219.
UA: Mozilla/5.0 ... Now multiple (see below).
URI: robots.txt ... Now ignored.

All IPs route back to "The Procter And Gamble Company", P&G, which has a massive number of IPs: 143.1.0.0 - 143.40.255.255 (2,621,440). All hits have been well-behaved and heeded robots.txt. Until today.

Today they became a fellow traveler with what I'm calling "Stub" because apparently that's what it is, an APNIC-STUB that was: "Transferred To the Ripe Region On 2025-05-14T08:37:17Z". So much for transparency.

Point is, Stub's related IPs, hitting with P&G's, ask for and receive robots.txt, but totally ignore it. So beware of the above, and of:

IP: 222.167.251.
UA: Mozilla/5.0
PLUS:
UA: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/138.0.0.0 Safari/537.36
UA: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.5735.199 Safari/537.36
UA: Mozilla/5.0 (Macintosh; Intel Mac OS X 13_4_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.5735.199 Safari/537.36
URI: robots.txt... Now ignored.

Both CIDRs are undeterred by UA-specific 403s. They just flip to a new (or old) ID. Time to block into the Chrome/140s now. (sigh)

[SumGuy]

143.20.219.0/24 is showing as being part of AS199959, AS834, AS7029 and AS398465. AS834 is IPXO so that's all you need to know, the other ASN's are just as botty.

222.167.251.0/24 is AS7029 (Windstream) and AS398456 (Rackdog). Again, both are full of bots. I could check my IP blocking list but I know I've blocked these entire ASN's.

I don't know where you're seeing these IP's associated with P&G but I'm sure that info dates to the 1980's and is no longer valid.

[Pfui]

That danged IPXO. I block them whenever I see them but their Ranges are seemingly bottomless (as well as bottom-feeding), as bad as contaboserver's. FWIW, I get my IP info from myip.ms. They've been reliable for me thus far, and the info loads quickly and cleanly (unlike, oh, domaintools). But I never notice dates, and wow, I should: "Whois Record Updated: 13 Jul 2022"!

Semi-Aside: Which site do y'all prefer for lookups?

[SumGuy]

> Which site do y'all prefer for lookups?
bgp.he.net

[lucy24]

Currently I use whois because I got fed up with domaintools kicking me out after some not-at-all-excessive number of searches. What I haven't found yet is a good lookup for IPv6 addresses--one that includes the CIDR so I know what to block.

[SumGuy]

What do you think you would lose if your server was IPv4 only (mine is) ?

[lucy24]

The whole server? How retro. My primary site is IPv4, but my test site and personal site--all in the same userspace on the same server--are IPv6 so I do occasionally get IPv6 robots.

[Pfui]

myip.ms sometimes includes IPv6. (But as I learned above, be sure to check the dates.)

[blend27]

@Phui -- That danged IPXO. --

They have an IP4/6 feed here: [geofeed.ipxo.com...]