bidswitchbot

[keyplyr]

UA: bidswitchbot/1.0
Protocol: HTTP/1.1
Robots.txt: Yes but using the UA: python-requests/2.12.5
Host: googleusercontent.com
35.192.0.0 - 35.207.255.255
35.192.0.0/12

ads.txt retriever for unknown source

[MitchNginx]

I've seen this one too also digging for ads.txt I wonder if this has anything to do with it? [iabtechlab.com...]

[keyplyr]

Yes, as I said above it is requesting ads.txt.

If you publish any type of ads you should have an Ads.txt [webmasterworld.com]

[MitchNginx]

Ok thanks keyplyr+ I was not aware of that, strange Google Adsense has never alerted me to creating such a file.

[keyplyr]

Here is what Google says about ads.txt:
[support.google.com...]

I see this file requested every day by an increasing number of bots.

[MitchNginx]

Awesome thanks keyplyr+ have now implemented their basic ads.txt on all my adsense sites. Now the mystery of all these bots looking for ads.txt is finally solved :)

[keyplyr]

Good work

[lucy24]

I see this file requested every day by an increasing number of bots.

So do I, and I don’t even have ads. Mostly the Googlebot, though. Are they afraid I’m using someone else’s ad system, since they know I’m not using theirs?

:: detour to raw logs ::

Oh, why there it is. But only on 1 April; after racking up a fistful of 403s, bidswitchbot (does the name mean “we’ll make you a better offer”?) seem to have given up and left. Aside from Googlebot, the others are all unknown or unnamed robots. And, since ads.txt doesn’t have the special exemption robots.txt does, the non-Google robots one and all come away with a 403.

Yes but using the UA: python-requests/2.12.5

Aside from the bidswitch UA switch (haha), I find this UA way back in October requesting both robots.txt and ads.txt. Same IP as when it used the bidswitch name, so perhaps an early incarnation of the robot. (October? srsly? The googlebot didn't start requesting ads.txt on a regular basis until February. Does bidswitch know something the rest of us don’t?)

Would Google ask less often if I gave them a blank ads.txt, by analogy with a blank robots.txt?

[MitchNginx]

@Lucy24 certainly something that has me thinking so tomorrow I will set up a special site just to track this behavior

[keyplyr]

The file may be requested using a variety of scripting languages and UAs.

It will be requested by Advertising Platforms, Marketing Companies, ISPs and others that have an interested role in the security and privacy of ad driven revenue.

Getting requests for this file is irrelevant if you do not publish ads. Simply ignore this activity.