Can't find apache log entry for visitors from bitly

[born2run]

So I'm a bit puzzled here, I have posted links using bitly service to the social media sites like FB and Twitter.

Now the hits on the bitly link (according to bitly website analysis page) keeps going up from only Singapore and nowhere else (like thousands!) but I don't find the corresponding hits in the apache access_log file!

I'm trying to locate the ip that's increasing my bitly hits.. it's probably a bot. What do i do now to investigate further?

[keyplyr]

Not all requests include a referrer any more. Many sites use "Referrer-Policy: no-referrer" (as does FaceBook) and HTTPS does a good job of removing referrers anyway.

I agree that it doesn't help us webmasters track our traffic, but such is life on the internet these days :(

[born2run]

I did a scan by the corresponding url address (of the bit.ly link) too but the hits are not there in the access_log file. How so?

[keyplyr]

As I said above about referrers...

The "bit.ly" part is a referrer because it forwards to the actual address at your server.

"bit.ly" sometimes does appear as referrer but not from Facebook (for the reason I explained above)

[keyplyr]

More...

Traffic following a any type of link at Facebook (including bit.ly) to your site will display in your Apache server access log as:

www.facebook.com/externalhit_uatext.php

As the referrer.

[phranque]

i think born2run is saying that if he sees N hits for a bitly url on bitly's website analysis page, he should see N+ hits for the redirect url in his server logs.

if it's a bot hitting bitly, perhaps the bot isn't following the 301 redirect to your server.

[born2run]

Yes phranque that was my point. So the bit hits bitly and then doesn’t visit my website url?

[lucy24]

If it is a bot: Do these spurious bitly hits have any real-world consequences for you (like money that you later have to return, or payments that have to be recalculated), or is it just one more irritation?

[born2run]

lucy24 I need correct stats on how popular my links are.. that's what I meant. Anyways I've stopped using bit.ly or any shortening service since twitter increased char count in tweets, and also as per keyplyr's advice here.

[keyplyr]

I need correct stats on how popular my links are

Then don't use bit.ly, use your real links so you can glean that data from your logs. If you need a way to do that, you can use Analog [mirror.reverse.net] which is free, accurate & highly customizable web log analysis software.

And you'll not ever be able to get accurate traffic stats from visitors following links you place at either Facebook or Twitter.