I've blocked Empty Header visitors, how to whitelist Facebook bot?

[born2run]

Hi so I've blocked empty header visitors via Cloudflare. I'm seeing it also blocks the legit Facebook bot as well. Is there any way I can whitelist this FB bot in Cloudflare? Please advise. Thanks!

[phranque]

empty header visitors

please define.
any visitor sending a HTTP Request header with an "empty" header value?
or sending specific "empty" header(s) get blocked?

[born2run]

Yes any visitor sending a empty header value. It seems Facebook bot still has same feature? Please advise.

[phranque]

i would do more research about which headers are being sent with empty headers and then block specific empty headers, but not the headers that FB bots send with empty values.

[lucy24]

Which header? And did you mean that they send the header, but it's empty? That's a major pain, because a lot of techniques--thinking especially of various Apache mods--don't distinguish between "empty" and "not sent at all". (In Apache logs it comes through as "" vs. "-" but that only covers Referer and User-Agent.)

In the specific case of Facebook, I hope you don't mean they have gone back to sometimes not sending a User-Agent, as they did for a few years. It was annoying, because I had to make a slew of extra rules along the lines of

SetEnvIf Remote_Addr ^66\.220\.1(4[4-9]|5\d) !noagent

Matter of fact, I see the rules are still present in my htaccess. There are a total of six, because in htaccess I think it's less work for the server than a single long and complex RegEx to cover all five IPv4 and one IPv6.

[born2run]

Ok thanks a lot guys. I appreciate the tips.