Hotlink Protection from Sites Blocking Refers on Cloudflare

[3zero]

Hello

Thought I might share this script and if possible perhaps improve it with community input

OK So first of all the problem, most image hotlink scripts rely on the the "referer" to block the image, however more advanced hotlinking simply removes the referer

So why not block blank referer? Well yes thats kinda what you have to do with an exception for searchbots

I use Cloudflare and have now added the following javascript to all pages using their new "Workers" feature and applied it to the image directory

I also think this could be used to protect other areas of the site (like forms)

addEventListener('fetch', event => {
 event.respondWith(fetchAndApply(event.request))
})

async function fetchAndApply(request) {
 let referer = request.headers.get('Referer')
// check if there is a user agent 
 if (request.headers.get('user-agent')) {
// if a search engine allow request
 if ((request.headers.get('user-agent').includes('googlebot')) || (request.headers.get('user-agent').includes('msn')) || (request.headers.get('user-agent').includes('yandex'))) {
 return fetch(request)
 } else {

//check if there is a referer
 if (referer) {
 // It's an image and there's a Referer. Verify that the
 // hostnames match.
 if (new URL(referer).hostname ==
  new URL(request.url).hostname) {
return fetch(request)
  } else {

   console.log('referer',referer)
  // Hosts don't match. This is a hotlink. Redirect the
  // user to our 404.
  return new Response('Sorry, this page is not available at this time.', {
  status: 404,
  headers: {
   'Location': '/404'
  }
  
  })
  }
 }
 }
 }
 else {
   console.log('referer',referer)
  // `No Useragent. Redirect to
  // 404
  return new Response('Sorry, this page is not available at this time.', {
  status: 404,
  headers: {
   'Location': '/404'
  }
  
  })
 } 
}

The reason I wanted to use a 404 as the error page is it gives the hotlinker the least information

[NickMNS]

@3zero your initial post was ambiguous you wrote:

I use Cloudflare and have now added the following javascript to all pages...

So it was unclear to me where you posted this code, and obviously based on the responses of others I was not the only one to be unclear on this point. As a result I posted the question:

Where exactly will this script be placed?

So I spent some time reading up on Cloudflares worker feature and as a result have a much better understanding of what exactly you are doing. To answer my own question the code is placed and run off of the Cloudflare servers (at the "edge"), any request for an image (in this case) will cause the code to be executed on the Cloudflare server and will either return the requested response or the alternate response (403).

@keyplyr wrote

The Cloudflare bot crawls the original site with the image and caches it. Once done, you won't see that many referrers from the hot-linking site, since requests for that image will now be pointed toward the Cloudflare caching system.

In this case this should not be a problem, because the code is being executed at the time of the request on the Cloudlfare's servers (at the edge between the User and Cloudflare) at this point the referrers, if present, should be available to the script.

But the fundamental issue raised by both Keyplyr and Lucy24 still remains and that is the issue of the false positives.

[Dimitri]

You can also simply make your site to write a cookie, each time a page is visited, with the time. Then, when it comes to serve an image, just check the value of the cookie, if the cookie is missing, then it's hot linking, if the cookie is present but the time is, let's say more than a minute ago, this is hot-linking too; etc... Just a thought.

[NickMNS]

@Dimitri the problem isn't blocking the hotlinking. The problem is blocking the hotlinking without preventing Google and other beneficial bots from accessing the images.

[Dimitri]

without preventing Google and other beneficial bots from accessing the images.

Then you just don't do the cookie test for allowed UA / IP pair.

[Qaeron]

The final script posted works fine NickNMS is deliberately misleading the forum

@3zero your initial post was ambiguous you wrote:
I use Cloudflare and have now added the following javascript to all pages...

Not true NickNMS is lying, here is what was actually wrote

I use Cloudflare and have now added the following javascript to all pages using their new "Workers" feature and applied it to the image directory

Next Lie NickNMS quotes Keyplyr and claims this issue isn't tackled

The Cloudflare bot crawls the original site with the image and caches it. Once done, you won't see that many referrers from the hot-linking site, since requests for that image will now be pointed toward the Cloudflare caching system.

However the cure for this in was posted again NickNMS choose to lie and ignore this fact

Code to remove existing cached images on cloudflare (can be removed after 1 month) for http.conf


<If "%{HTTP_REFERER} =~ /yourdomain/">
</If>
<Else>
<FilesMatch "\.(ico|pdf|flv|jpg|jpeg|png|gif)$">
Header set Cache-Control "private, no-cache, max-age=0"
Header set Pragma "no-cache"
</FilesMatch>
</Else>

Nick NMS implies false positives another lie

Nick NMS then claims the script blocks Google and legitamet bots another lie

The problem is blocking the hotlinking without preventing Google and other beneficial bots from accessing the images.

He ignores the extendable code that not only deals with search bots but social networks too

if ((request.headers.get('user-agent').includes('googlebot')) || (request.headers.get('user-agent').includes('applebot')) || (request.headers.get('user-agent').includes('bingbot')) || (request.headers.get('user-agent').includes('pinterest')) || (request.headers.get('user-agent').includes('facebookfacebookexternalhit')) || (request.headers.get('user-agent').includes('facebook')) || (request.headers.get('user-agent').includes('twitter')) || (request.headers.get('user-agent').includes('GoogleImageProxy')) || (request.headers.get('user-agent').includes('yandex'))) {
return fetch(request)
}

Disgarcefull Behaviour from NickNMS!

[Qaeron]

PS mods you can now close this account I want nothing more to do with webmasterworld!