htaccess ban referer via wap

[acegames]

Hi , new to the forum and first post , thought someone on here might be able to help with this one .

Got a wap page I have set up , proving to be popular , thing is people keep hotlinking to my content from their wap pages , thus stealing my bandwith .

mobile phones don't send a header to the server so are basically undetected , so I am finding it hard to stop the linking .
Is there a way around this so browsers can only download if they are refered from my webspace?
maybe configure server to "request" header from the mobile phone .

Hope someone can help , I lost 30 gig of transfers last month!

Regards Dave

[wkitty42]

welcome to webmaster world!

doesn't the standard .htaccess methods of blocking hotlinking work with WAP pages? i believe it should... take a look at the threads or search google for

site:www.webmasterworld.com hotlinking

you'll find quite a few messages on it... i just did a quickie and got about 500+ from google ;)

[acegames]

No thats the point , I can manage it with all the web browsers but not wap .

The phones dont send any referer data to save bandwith , therefore I cant block the referer via wap only web .

[bcolflesh]

They may not send a referrer, but most WAP-enabled devices send a user agent (usually with the word WAP in it) - block on user agent and then by IP for the stragglers.

[acegames]

I need to block referers coming from a certain url only if I block from the user agent it would block all the legit users of my site too .

[wkitty42]

how can you tell that the traffic is coming from hotlinks? that would be the key to blocking them... can you show a section of your logs that shows one of these hotlinked accesses and a normal one from your site? make sure you replace the domains with examp1 and examp2

[acegames]

Thats it my logs show no referer from mobile phones , just the phone

[jdMorgan]

> thing is people keep hotlinking to my content from their wap pages , thus stealing my bandwith.

wkitty42 makes a good point - How can you tell this is "hotlinking" given the nature of WAP and the fact that heavy caching is used, etc.? There is no information avaialable, except the behaviour pattern of the requests to indicate whether this is hotlinking or not. However, for the sake of conversation, let's assume that it is hotlinking. What can be done?

For situations like yours, a workable approach might be to use dynamic links for the objects included on your pages such as images, videos, music clips, scripts, etc. Changing the links once a month -- or even once a day -- prevents long-term hotlinking. The link-changing can be done with a script or with mod_rewrite and Server-Side-Includes. The only trick is to pick a method - an algorithm - that allows the just-made-stale links to continue to work for the duration of time that you have specified in the pages' Expires-after header.

So, for example, you could put the numeric date into each link, in the form "/images/20040213/logo.gif" and then set up a redirect so that all files requested from /images/20040212/ continue to work for an additional 24 hours as well. This would work if you set your expires headers to 24 hours. However, all hotlinks would stop working after 48 hours.

You could set an environment variable in .htaccess with the date string (lets call it "new_link"), and then use that string both in the stale-link rewrite code and in SSI to include it on each page. So, your links would look like:

<img src="/images/<!--#echo var="new_link" -->/logo.gif"> border="1" height="60" width="120">

and in .htaccess:

RewriteRule .* - [E=new_link:20040213]
RewriteRule ^images/20040212/(.*)$ /images/20040213/$1
RewriteRule ^images/20040213/(.*)$ /images/secure/$1 [L]
RewriteRule ^images/[0-9]{8}/ - [F]
#
ExpiresDefault A86400
Header unset Cache-Control:
Header append Cache-Control: "must-revalidate"

Then change those four RewriteRule lines (very carefully) once a day -- I suppose you could even automate that, too. If you miss a day, you risk only an extra day of successful hotlinking - nothing else will "break."

The subdirectory "/secret" is simply the "real" subdirectory where you actually store your images, and will not appear in any links. Nor will it be visible to search engines. You may want to make allowances for your search-engine cached pages to show images properly if a referer is available, and that can be done by redirecting all know-good search engine cached-page referrals to the proper image directory, for example using the code:

RewriteCond %{HTTP_REFERER} ^http://.*(search¦cache)
RewriteRule ^images/[0-9]{8}/(.*)$ /images/secure/$1 [L]

inserted above the code shown above. This allow-cached-page-referrals code is not be all-inclusive -- it's only intended as an example. You cannot allow blank refererrs in this code without opening yourself up to the original problem.

I haven't implemented or tested this, but maybe it'll give you food for thought. There are certainly more sophisticated solutions available, but I sense that your problem is immediate.

The most important thing to "get right" with methods like this is that you must continue to support the "old" links until all caches containing that old links have expired as specified by your Expires settings. Otherwise, your site will look broken, even to normal visitors, if their browsers hold an unexpired cached copy of the page linking to the images. :(

It's a complex subject, and the details depend on how you want to implement it, but that's just an example of one work-around for your problem. It is not perfect, and it can be bypassed by a determined hotlinker who guesses your link-generation method or your "real" image subdirectory name. But you don't have to make it impossible to hotlink, you just need to make it difficult, and they'll hotlink elsewhere.

Jim

[acegames]

wow , thanks for that reply .
I am 100% being hotlinked to , I am able to do a search on a certain page to see exactly who and what files are being linked to , an average of 100 wap pages are directly linked to my content .

As for your idea sounds good to me but would require a lot of rework with the wap page , I have over 1000 files for download so I would prefer something a little easier ;)

Changing back to my original idea , although the phone does not send a referer header could I not set up my server to request one from the phone or does the phone not hold this information?

Thanks for all the input on this one!
Dave

[acegames]

this is a small portion of my log

[13/Feb/2004:17:57:53 -0600] "GET /members/Pictures/Pictures.php HTTP/1.1" 200 1898 "-" "Nokia3510i/1.0 (04.44) Profile/MIDP-1.0 Configuration/CLDC-1.0 UP.Link/5.1.1a"
193.113.200.166 - -
[13/Feb/2004:17:57:53 -0600] "GET /members/Polyphonic/SpaceMan.mid HTTP/1.1" 200 7155 "-" "Nokia3650/1.0 SymbianOS/6.1 Series60/1.2 Profile/MIDP-1.0 Configuration/CLDC-1.0 UP.Link/5.1.1a"
193.35.129.169 - -
[13/Feb/2004:17:57:53 -0600] "GET /members/Polyphonic/kickstart.mid HTTP/1.1" 200 2004 "-" "Nokia7250I/1.0 (3.22) Profile/MIDP-1.0 Configuration/CLDC-1.0"
216.155.165.240 - -
[13/Feb/2004:17:57:53 -0600] "GET /wappage.wml HTTP/1.1" 200 491 "-" "SEC-SGHX105 NW.Browser3.01"
217.42.38.19 - -
[13/Feb/2004:17:57:55 -0600] "GET /Forum/images/smilies/standart/exclamation.gif HTTP/1.1" 304 - "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
193.113.200.148 - -
[13/Feb/2004:17:57:55 -0600] "GET /members/Pictures/ngage/bond007.gif HTTP/1.1" 200 9917 "-" "NokiaN-Gage/1.0 SymbianOS/6.1 Series60/1.2 Profile/MIDP-1.0 Configuration/CLDC-1.0 UP.Link/5.1.1a"
217.42.38.19 - -
[13/Feb/2004:17:57:56 -0600] "GET /Forum/images/smilies/standart/wink.gif HTTP/1.1" 304 - "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
217.42.38.19 - -
[13/Feb/2004:17:57:56 -0600] "GET /Forum/images/smilies/standart/shocked.gif HTTP/1.1" 304 - "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
193.35.129.169 - -
[13/Feb/2004:17:57:57 -0600] "GET /members/Polyphonic/Rainbow.mid HTTP/1.1" 200 8669 "-" "SIE-A60/10 UP.Browser/6.1.0.5.c.6 (GUI) MMP/1.0"
212.183.131.163 - -
[13/Feb/2004:17:57:57 -0600] "GET /members/Polyphonic/latest2.php HTTP/1.1" 200 2691 "-" "Nokia6220/2.0 Profile/MIDP-1.0 Configuration/CLDC-1.0"
193.113.200.204 - -
[13/Feb/2004:17:57:58 -0600] "GET /wappy/logos/logos.php HTTP/1.1" 404 555 "-" "Nokia6220/2.0 Profile/MIDP-1.0 Configuration/CLDC-1.0"
217.42.38.19 - -
[13/Feb/2004:17:57:58 -0600] "GET /Forum/images/smilies/standart/jump_crazy.gif HTTP/1.1" 304 - "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
217.42.38.19 - -
[13/Feb/2004:17:57:58 -0600] "GET /Forum/images/smilies/standart/jumping_smiley.gif HTTP/1.1" 304 - "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
217.42.38.19 - -
[13/Feb/2004:17:57:59 -0600] "GET /Forum/images/smilies/standart/question.gif HTTP/1.1" 304 - "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
193.35.129.161 - -
[13/Feb/2004:17:57:59 -0600] "GET /members/sms/final_fantasy_sms_1.MID HTTP/1.1" 200 993 "-" "SAMSUNG-SGH-E700/BSI UP.Browser/6.1.0.6 (GUI) MMP/1.0"
193.113.200.204 - -
[13/Feb/2004:17:57:59 -0600] "GET /wappy/logos/style.css HTTP/1.1" 404 555 "-" "Nokia6220/2.0 Profile/MIDP-1.0 Configuration/CLDC-1.0"
193.113.200.236 - -
[13/Feb/2004:17:58:00 -0600] "GET /wappy/Polyphonic/IceIce.mid HTTP/1.1" 404 588 "-" "Nokia3510i/1.0 (04.01) Profile/MIDP-1.0 Configuration/CLDC-1.0 UP.Link/5.1.1a"
217.65.192.33 - -
[13/Feb/2004:17:58:00 -0600] "GET /members/Games/series40/series40.php HTTP/1.1" 200 1166 "-" "SonyEricssonT300/R101"
193.35.129.161 - -
[13/Feb/2004:17:58:00 -0600] "GET /tunes.wml HTTP/1.1" 404 530 "-" "SIE-A55/07 UP.Browser/5.0.3.3.1.e.4 (GUI)"
66.36.200.253 - -
[13/Feb/2004:17:58:01 -0600] "GET /Pictures/boselectas/faces.gif HTTP/1.1" 404 570 "-" "Nokia3510i/1.0 (05.00) Profile/MIDP-1.0 Configuration/CLDC-1.0"
193.113.200.166 - -
[13/Feb/2004:17:58:04 -0600] "GET /members/Samsung/Samsun.php HTTP/1.1" 200 4206 "-" "SEC-SGHP400 UP.Link/5.1.1a"
193.113.200.153 - - [13/Feb/2004:17:58:05 -0600] "GET /members/Pictures/angelina/angelina.php HTTP/1.1" 200 636 "-" "Nokia3510i/1.0 (04.44) Profile/MIDP-1.0 Configuration/CLDC-1.0 UP.Link/5.1.1a"

[jdMorgan]

Servers serve, and user-agents make requests. The easiest way to think of this is that servers do not speak unless spoken to. Therefore, there is no way to have your server "ask" for anything.

You could set something up requiring a cookie to be set by your home page(s) before serving any images, but be aware that some percentage of surfers disable cookies, so again, your site will look broken. Like blocking blank referrers, this is a partial solution, but it causes customer service headaches... How many e-mails a day do you want from people telling you that your image links are broken? Even if you put up a page telling them that cookies are required, this will cause many of them to leave, either because they don't want you telling them they have to enable cookies, or because they don't know how.

The method I described above is the simplest of the "passive" methods I could think of.

Jim

[acegames]

Thanks for that , at least I understand more about not being able to "request a header"

[Martink]

This might be possible :) I have two ideas!

Firstly, if WAp can accept cookies, thats easy just plonk one on and only let them access if the cookies is there.. Secondly...

You are given the user agent, and you can block based on user agent. Instead of using a blacklist, is it possible to use a white list!?

That way, when somebody accesses your wap page, you record their user agent and add this to your white list and make it so that images can only be accessed by those in your whitelist :)

Ok, there are problems with this, for example the quantity of data stored on the whitelist and the fact that some peoeple may visit your site once and then other sites many times. However! If you reset the whitelist every 24 hours via cron which is easy, or even better you delete the user agent after 24 hours using a script, e.g. PHP then you would effectively be giving them on a session on your server even though you can't use cookies.