Forum Moderators: open

Message Too Old, No Replies

Does cloudflare hosts bots / scrapers?

Is it normal to get web-hits from cloudflare IP's?

         

SumGuy

4:01 pm on Oct 13, 2022 (gmt 0)

5+ Year Member Top Contributors Of The Month



I'm seeing an up-tick today from cloudflare IP's hitting my web server. I have to check my router blocking rules but I believe I'm blocking this particular /16.

Is there a legit use-case to expect organic hits (ie human browsers) to come from cloudflare IP's?

Pfui

4:02 pm on Nov 30, 2022 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I've been wondering about Cloudflare generally myself, since 2015:) Thing is, some hits seem like real visitors, but the vast majority nope. For example, in the last few weeks, here are some probable fakers based on the unknown (to me) UA -- but who knows?

172.71.178.8x - rn3/4406 CFNetwork/1390 Darwin/22.0.0
172.71.118.7x - rn3/4406 CFNetwork/1333.0.4 Darwin/21.5.0
172.70.162.18x - rn3/4303 CFNetwork/1325.0.1 Darwin/21.1.0
172.70.85.24x - rn3/4302 CFNetwork/1312 Darwin/21.0.0

So I keep blocking on UA, and watching. And blocking the likes of this semi-oldie:

Mozilla/5.0 (compatible; Cloudflare-AMP/1.0; +https:// amp. cloudflare. com/doc/fetcher.html) AppleWebKit/534.34

jmccormac

11:46 am on Jan 13, 2023 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I've seen some scrape attempts from Cloudflare IPs and have blocked them at the IP level. The attempts appeared to be sporadic rather than sustained. The UAs were browser UAs.

Regards...jmcc

Sgt_Kickaxe

12:05 pm on Jan 13, 2023 (gmt 0)



Cloudflare can do it all, including checking and blocking your visitors when YOU don't use Cloudflare on your website.

Seriously, there are apps, plugins and other useful services like ad partners and CDNs who pass traffic through Cloudflare without the owner being aware. It's worse when it goes through AWS, too. When that happens EVERY trusted metrics company gets a ride on your site, to help YOU, of course.

Always control the DNS if you want to be in charge of user experience, and know how 3rd party stuff 'helps' you.