visionutils

now in the wild

[keyplyr]

UA: visionutils/0.2
Protocol: HTTP/1.1
Robots.txt: No
Host: Various

As discussed in earlier thread [webmasterworld.com...] this is the image caching library at FB that engages when an image accompanied link gets posted or shared. I believe it may also re-validate images at a later date.

*However* lately I've been seeing this agent come from non-Facebook IP ranges. It can be downloaded from GitHub making it available to anyone.

I only allow it from FB IP ranges.

[aristotle]

It seems like facebook is always fetching the same image over and over and over again.

[lucy24]

facebook is always fetching the same image over and over and over again

It looks as if it fetches the image each separate time someone views the page.* Currently they do it without a referer, so it bypasses ordinary hotlink protection; you have to look at the IP. From this, I deduce that most people who post about something of mine do not have many friends ;)

I also learned recently that this link-posting business can be done by direct type-in, with concomitant possibility for typos. (I got facebook requests for example.com/url-that-really-exists-on-example.ca. This really baffled me until I figured it out. At first I thought they were looking for a trackback type thing, and had no idea why they'd expect to find it.)


* Not long ago I found something analogous happening at reddit: zillions of requests for the favicon only. Individual user's IP and UA, with the reddit thread as referer. It's a plugin. Presumably it should be encouraged, because it makes the link more attractive-- rather like Firefox's Favicon Reloader-- and hence makes people more likely to click on it.

[keyplyr]

Currently they do it without a referer, so it bypasses ordinary hotlink protection

Well that's the new normal (Google,Bing, et al) but since it is not *our* server that is serving the endless image views at FB, hotlinking doesn't actually happen (I know you know that... just sayn')

FB's constant redundant hits to our image files are probably caused by their caching schema. When you consider the >millions of user posts every hour, their caching utility would need to consistently purge, thus come back for another copy periodically.

[lucy24]

just sayn'

Yes, I guess there's no technical term for "activity on someone else's site causes work for my server".

<topic drift>
Speaking of referers, I found another good typo header: Referrer. In fairness to whoever wrote the script, I have to concede that misspelling something you know how to spell properly is almost as hard as intentionally mispronouncing a word (like the various countries' names for each other's major cities).
</td>

[keyplyr]

Guess I don't see this as an unrewarded burden from FB.

The various image search engines that no longer include a referer always has bothered me. I get *some* traffic but nowhere near what it was when I was able to execute a rewrite forcing a parent url :)