Forum Moderators: open

Message Too Old, No Replies

SiteOne-Crawler

Is this a known bot/crawler?

         

SumGuy

2:07 am on Jan 25, 2024 (gmt 0)

5+ Year Member Top Contributors Of The Month



This came in recently from 89.233.159.231 (AS12767, T-Mobile Czech Republic).

Mozilla/5.0 (compatible; SiteOne-Crawler/1.0.7; +https: // crawler.siteone.io/bot/?r=8-character-string)

It appended ?checkedBy=crawler.siteone.io to it's request for my landing page.

I had a look at the site, I don't think it's supposed to be crawling random sites without being under user-direction, maybe I'm wrong about that. I've decided to block it - it doesn't seem useful.

tangor

4:32 am on Jan 25, 2024 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Since I block by .io I haven't specifically noticed this one as I don't spend a lot of time looking at the 403s in the log (filtered out).

Just looked at January so far, the above has not appeared.