Forum Moderators: open

Youbot (new crawler?)

and google-inspection tool

         

SumGuy

11:50 pm on Mar 13, 2026 (gmt 0)

5+ Year Member Top Contributors Of The Month



Saw this UA today:

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; YouBot/1.0; +https://docs.you.com/youbot; env:prod) Chrome/142.0.0.0 Safari/537.36

From AWS 32.192.232.84 and 32.192.6.227

IP's have generic AWS host-name (various.compute-1.amazonaws.com). It's not doing itself any favors by using such a wide range (or wide gap) of IP's. And it doesn't have it's own host-name. Have scanned logs, no previous instance of "youbot" ever seen.

It only grabbed two interior HTML files, no idea how it knew their URI before-hand. It did ask for robots.txt before it grabbed each file.

Also saw this today:

Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.199 Mobile Safari/537.36 (compatible; Google-InspectionTool/1.0)

From 24.187.105.91 (cablevision IP, not a legit google IP).

I checked the logs for previous instance of "inspection tool" and found this:

Mozilla/5.0 (compatible; Google-InspectionTool/1.0;)

Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.5735.179 Mobile Safari/537.36 (compatible; Google-InspectionTool/1.0;)

From July 2023 (from 66.249.72.173 which is a legit google IP). And this:

Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.6167.85 Safari/537.36 (compatible; Google-InspectionTool/1.0)

From Feb 2025 from 47.132.74.82 (Spectrum cable, not a legit google IP).