homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Visit PubCon.com
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

facebookexternalhit - Accessing /images/ now?

 5:35 pm on Nov 10, 2012 (gmt 0)

I know there are a number of threads for this UA already, however, they are too old to respond to: [webmasterworld.com...]

Safe to say a lot of us are being visited by user-agent: facebookexternalhit/1.0 [facebook.com...] -- correct?

How many of these hits are simply trying to access all of your entire /images/ folder - no content specified?

A few IPs:

All visits from these varying Ips are within a few minutes of each other.



 1:01 am on Nov 11, 2012 (gmt 0)

your entire /images/ folder - no content specified

Do you mean that they're requesting
like that? They're hoping to find an index file-- probably auto-generated-- that will helpfully provide a list of all images in one handy location. Do not oblige them. (This is generic advice for all image directories everywhere, unless you've got some specific reason to do otherwise.)

btw, those are all the same IP. Well, two IPs.

:: off to pursue line of thought ::

Nope, nobody's been snuffling around any /images/ directory lately. I tend to auto-ignore 403s, which is what they'd get, so I had to go back and check semi-manually.


 2:28 am on Nov 11, 2012 (gmt 0)

I've seen it a lot lately. We don't allow a directory listing view of our site, but I've seen them almost in a spidering style. One right after the other.

I have a little FB widget that shows what pages have been "Liked" or shared and interestingly enough those pages or images that they are accessing aren't shown as a human "like"


 3:25 am on Nov 11, 2012 (gmt 0)

Yes, all of /images/ and it is fed a 403 on each attempt.

Bewenched, I haven't noticed full-fledged spidering activity yet, but I do wonder what they're up to. Perhaps an image search engine in the works?


 11:08 am on Nov 11, 2012 (gmt 0)

How many of these hits are simply trying to access all of your entire /images/ folder - no content specified?

In the 2 years I've been using FB as a traffic generator, I have never seen an authentic FB UA attempt access to an image directory without the specific image file used on the page... and I keep a very tight lookout on my daily logs, especially from major traffic sources.

I'm guessing this is one of the APs added by FB users (pinterest, simcity, etc) which would all get a FB IP and UA (facebookexternalhit/) since FB controls all activity from within its UI.

However, this may be something new I haven't seen yet. If it happened at my site, I'd complain to FB.


 8:50 pm on Nov 25, 2012 (gmt 0)

Well facebook did announce they are working on a search engine with yahoo I think.

These could also be someone hotlinking the image inside one of those dreaded facebook note spam amazon affiliate things I've been seeing lately.


 9:36 pm on Mar 28, 2013 (gmt 0)

This topic is fairly old, but by accident I discovered the cause of my errors:

I started using Facebook's OpenGraph (to display a chosen picture, rather than have them scan the site for a pic and let the poster choose a thumbnail) - on pages I dynamically set, I forgot to add an:
if (isset($_GET['whatever'])) on pages I didn't have a chosen thumb specified, so it was grabbing /images (which obviously, triggered errors, because I had http://www.example/com/'.$variable.' set)

[edited by: bill at 8:38 am (utc) on Mar 29, 2013]
[edit reason] unlinked example [/edit]

Global Options:
 top home search open messages active posts  

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved