One entry in my robots.txt
is shown below, yet today, two Google IP addresses (220.127.116.11, 18.104.22.168) visited [u]only[/u] two files in a directory named /jscript/ .
There was no Google-related U-A in the log entry (Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:22.214.171.124) Gecko/20060909 Firefox/126.96.36.199)
, but shouldn't Google be adhering to the below rules regardless of the IP or U-A that they use?
Thanks in advance.