large number of visitors from "Facebook"

referer, abnormal traffic, bots, facebook,

[Francies]

Not sure this is the correct forum, so please tell me... since 3 days, I have a large (huge for my website) number of visitors on a single webpage, most with referer "m.facebook.com" and a few % "www.facebook.com".
At first I was happy, even updated the page, then I asked on my page which group was linking to my website, as there is a form to leave comments, but no answer...
So I became suspicious, I had a look at the addresses, and while user agents and referer are obviously mobile, IP's are looking more like PC's...
So without human responses and this "mobile from fixed addresses" issue, I suspect they are not something positive... I redirected them.
I'm wondering what their goal would be ? I fear they could mess with my google ads, and I already had a few ads related bots visiting the same page, like they want to check something !
It's the first time it happens on my website, although I have external referer from time to time (facebook included), they usually come in numbers, but then decrease rapidly, here they happened suddenly, but after 3 days the number was still high. The hours and country of origin correspond to the normal visitors, though, more or less.
Anyone heard of this ?

[TorontoBoy]

Are you checking your raw access log? Take their IPs and look them up at whois.com. If they are not FB then ban them in your htaccess. Most of the popular sites are faked by bad bots wanting to evade detection. m.facebook.com is pretty common.

[lucy24]

For every visitor coming in from facebook, logs should show at least ten times that many requests for a single image (whichever one was selected by the human who originally “liked” the page). Everyone sees the link; not everyone follows it.

Mobile users from a static IP can be perfectly legitimate. They're using their home WiFi because someone else is on the desktop--or because they themselves don't feel like sitting in front of a screen--having established that, for one reason or another, this yields better results than using the phone's data connection.

[TorontoBoy]

Here is the user agent UA name that hits be daily:

facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)

All the IP with this UA I do verify come from FB

[phranque]

The most legitimate reason for a mobile device to be on a WiFi IP instead of their mobile provider's IP is to save their mobile data plan usage for when they are actually mobile.

[keyplyr]

Hello Francies and welcome to WebmasterWorld [webmasterworld.com]

I wouldn't "redirect" these requests until you understand what's really going on.

1.) "facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php" is the UA from Facebook BUT Facebook also requests files using a blank UA ("")

2.) Any UA, from anywhere, using any IP can follow links found on Facebook to visit your site. These are not necessarily malicious agents. But it does occur.

3.) Mobile UAs using non-mobile ISP ranges, WiFi and other connectivity is not abnormal (as phranque said.) I see it all the time.

4.) Many visitors coming from FB will be using the Facebook for Android or Facebook for iOS app and can use their home or office WiFi with their phone. This is extremely common.

[Francies]

Thanks a lot for your advice ! I'll answer to the latest post, which list the other points :

1. these are "regular" agents, not the FB UA (which also hit before the bots) that also visits other pages from time to time
2. agree
3-4. I understand this point, although in this case the mobile percentage is not natural for my website, like 99% mobile/PC

I edited the page removing all text except an alert asking to post a message with the original page/group linking to my url, and after hundreds of visitors NOBODY posted anything ! I then redirected to some non existent domain yesterday evening, and when I removed the redirection this morning, the bots were still hitting the same way ! No human reaction to correct the link on the Facebook post ? To visit the page again and answer my question ?

These are clearly 99% bots, the fact that the FB UA hit before the flow started, could mean it's some automated Facebook marketing campaign - a few bots related to advertisers also visited this page since then - maybe the Facebook app is silently hitting pages unbeknown from the users, or theses PC have been hacked and something else than FB is behind this, but to me these are not humans, else after thousands of hits someone would have responded to my inquiry !

All signs point to some kind of sophisticated marketing campaign, maybe to scam advertisers, or hype some websites as referrers in the statistics ! Sounds far fetched, but after 4 days I see no better explanation.

[keyplyr]

Then it sounds like a bot-net. If you have all the server farm ranges blocked, you can ignore them.

[Francies]

Yes. But I can't block IP ranges, as they come from usual ISP's. They are not from China, Russia, or amazonaws servers - that's not a surprise I also can't use IP's when I have to block hacking attempts from PC's trying to scan known vulnerabilities, as many also come from normal ISP ranges.

I block the referrer in relation to the page they visit, that's the best solution as usually this page doesn't get much visitors anyway... Even if they are not "attacking" the site, they are useless, I have nothing to gain here. Junk traffic. Not good.

24hrs later, still no human to answer my request on this page ! and that's after thousands and thousands of views. Also I found another 600-700% discrepancy in comparison to my usual UA's stats.

Most likely trying to scam someone to sell the referrer "look at the huge traffic we have !", they are hacking their own statistics to pop up their advertisement prices. I can see no other use to this invalid traffic !

[keyplyr]

I can't block IP ranges, as they come from usual ISP's. They are not from China, Russia, or amazonaws servers - that's not a surprise I also can't use IP's when I have to block hacking attempts from PC's trying to scan known vulnerabilities, as many also come from normal ISP ranges.

That's what bot-nets typically are... compromised ISP accounts, or servers.

So, as I said, block the server farm IP ranges (not the individual IP addresses) and ignore the rest. The infected ISP accounts will presumably be fixed in time.

Most of this is routinely discussed in the Search Engine Spider & User Agent ID Forum [webmasterworld.com] and the blocable ranges are listed in the Server Farm IP Ranges thread [webmasterworld.com]

But just be aware, if someone (or you) placed a link to any of your pages or files at FB, FB itself will send a lot of requests to get all files associated with that link. This is normal and nothing malicious.

[Francies]

The only solution to prevent this junk traffic is to block referrers from m.facebook.com (only for a unique page now) !

Thanks for the links and tips, but blocking range here sounds difficult, really, I would have to block myself most likely ! At least a dozen countries, mine included, many different ISP's... and I have only checked around 50 IP's over maybe 15K by now ? can't remember seeing a pattern, I would have to manage hundreds or more ranges ! (much more than 15K invalid traffic in 4 days most likely, if I hadn't blocked the 99% m.facebook.com referrer on this url) ! Many of them are not fixed IP's, whether they are infected PC's pretending to be mobiles, or FB apps on real mobile phones hitting websites without the owner being aware of, in any case these users would use another address the next day when they switch their computer or phone on, so blocking the whole ISP's ranges would quickly become an issue for my regular visitors...

No issue with requests for images yet. I don't believe it's a real link, more like some hack or marketing operation, masquerading as real users.

Invalid, junk traffic. Can't remember seeing that.

[tangor]

I AM NOT SUGGESTING YOU DO THIS as there are others on WW that think any page getting traffic, good, bad or indifferent, is a good thing.

Back in the day (when bandwidth pricing was critical) my solution to the same problem described was to nuke it (the page) and 403, then create a replacement page and go from there. Bandwidth is not as critical these days, but the aggravation has not diminished. Some pages get singled out and hammered and there's no reason why, it just is.

You can grin and bear it, or you can nuke it and move on. Don't know of any other choices.

[Francies]

I may consider your idea of "nuking" that page... until then I have found another solution ;-)

See, I don't get any profit from this. It's on my website, I pay for that, and this traffic is "invalid", as the big guys say. Junk. Imagine someone comes and disposes waste on your yard, saying, it's a big yard, we noticed you never use or walk on these acres, so we will use them to make money, but we won't give you anything in return. That's what is happening here. I don't really care if it's a "good thing" for "others" ! Who are these "other" people to start with ? They come and talk to me, offer something in exchange ! Right !

If it was on a client's, well, I could consider it as more traffic, I would at least wait to be sure what this is about, exactly - although I'm already convinced it's not honest (did I mention that already ?)

BTW whenever I can, I already block these Ukrainian and Russian spammers trying to promote websites with faked referrers, for me this is the exact same technique, only a few thousands times more massive ! Neither the Russians nor Facebook offer me anything in return, why should I let them hack my stats to make more money ? They already have enough, I haven't. Really. It would be promoting injustice to allow that kind of scam to happen. When I'm wealthy and successful enough not to have that much time on my hands, and as long as it's no security issue, well, let them play or scam their clients with fake numbers ! Until then it's a liberty, mine !

And speaking about bandwidth, I suppose I'm not the only one getting hit by massive fake traffic, so imagine the energy burned to keep these thousands of servers up and running ? That CO2 thing, climate change, and all ? You heard about things getting warmer, didn't you ;-) ?

[lucy24]

Sometimes a redirect is the answer. One I currently use looks like this

RewriteCond %{HTTP_REFERER} ^https?://(l|m|www)\.facebook
RewriteRule ^games/(tower|palace|canal|color)/?$ https://www.example.com/games/online.html [R=301,L]

Explanation: certain pages use so many supporting files, I can actually tell by the size of the day's log file (in K) how many people have visited. Some of those are people who just blindly click every link they see, so it's taking up their time and my bandwidth for nothing. So when there's a request with a FB referer I redirect to a closely related page--using <10 instead of 100+ supporting files, and no scripts--where humans can stop and think about whether they really want to be there. And where robots, if any, miss out on the chance to learn a bunch of filenames.

[Francies]

Yes, that rewrite condition is my final solution, without the 301.

RewriteCond %{HTTP_REFERER} m\.facebook\.com [NC]
RewriteRule ^article_targeted_by_fb.php$ http://www.othersite.com?information [NC,L]

And not pointing to another of my pages, as it's not about server's resources here (at least it wasn't this time, maybe it's only the beginning), but some people might be very interested to learn about that junk traffic ? I understand from previous posts that most on this thread advise to keep these bots, not to block them or send them to h... but I don't see the point. Of course if I was selling something, I would want to believe that a few % of them may be real users...

More than 10K fake users on this page before I blocked the m.facebook.com referrer, how many websites in the world used for the same scam ? dozens of millions ? that's an "invalid traffic" of hundreds of billion in a few days ! Still no idea what's up yet, do they find a way to "sell" these fake visits, to convince potential clients ? are they trying to trick webmasters, too ? I had never seen something so sophisticated, there is also a very small amount of www.facebook.com and l.facebook.com/l.php?u=... referrers that I haven't blocked yet, and now even a few google plus referrers visiting the same page, which was only hit a few times a month at best !

Yet still no human reaction to my friendly request on that page to indicate the url/group with the incoming link (and there is no sign up/login required to post an answer !) 100% bots, but nearly impossible to detect with so many distinct IP's, UA's, etc. Some ratios are totally off, though, compared to other visits, even from FB !