homepage Welcome to WebmasterWorld Guest from 54.211.34.105
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Facebook ?
69.171.xxx.x
Bewenched

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 4508870 posted 1:15 pm on Oct 17, 2012 (gmt 0)

69.171.237.6

now normally I'd think it was just their little crawler off to grab the image for someone's "Like" post ... but there's close to 50+ sessions of it going on right now.

Gotta get more details when I get in the office.

 

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4508870 posted 7:15 pm on Oct 17, 2012 (gmt 0)

Facebook does not have a spider or crawler, just a fetcher. When a FB user wants to post a link to your site, the FB image fetcher searchers around on your page for an image to use as a thumbnail. It will grab 3 images and present all three for the poster to choose from. However the poster may cancel the post and rewrite the comment and try again. The image fetching process will repeat each time and may add up to many hits on your server.

Sometimes, if you're lucky, other FB users will re-post your link. Each time this is done, the FB fetcher will look around your page for thumbnails. They are not trying to steal anything from your site, only making the incoming link look better to users.

I allow all their IP ranges to grab my images, hence I get double (sometimes triple) digit daily traffic from FB.

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4508870 posted 11:14 pm on Oct 17, 2012 (gmt 0)

It's
69.171.224.0/19
And, sorry, keyplyr, but I don't get no three randomly selected images. When they come by, it's every single image on the entire page. It's nice that in your case it leads to real human visits to the actual page, but for some of us it's just wasted resources.

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4508870 posted 11:20 pm on Oct 17, 2012 (gmt 0)

but for some of us it's just wasted resources


Ditto, however not much waste when all they get is 403's ;)

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4508870 posted 12:19 am on Oct 18, 2012 (gmt 0)

Lucy, that's not what I said. I said it grabs 3. It "looks" at more than that, but only uses 3. The poster then can decide which one of those 3 to be used as the thumb when FB sends you all that traffic. Awful isn't it :)

And if you're not getting traffic from this, then you either do not have a page anyone is interested in visiting or you are blocking the human traffic somehow or blocking the image retrieval hence no thumbnail for anyone to click on and get to your page.

We've had this discussion twice before. You will never understand how this works until you open an account at FB and see it for yourself (which I recommend BTW.)

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4508870 posted 3:52 am on Oct 18, 2012 (gmt 0)

It "looks" at more than that, but only uses 3.

From the grabbee's point of view, this is a purely academic distinction.

I can only assume that there is a substantial overlap between the personality type that is drawn to FB recommendations, and the personality type that is drawn to the content of your site(s). This is not a universal truth that can be applied to all sites and all circumstances.


Only twice? It feels more like seven.

Edit:
I just went over to feed my raw logs through their ordinary processing, as I do every day or two. It is, of course, pure coincidence that I found them bloated beyond all reason-- largely thanks to FB trying repeatedly to pick up every single picture in my most image-rich e-book. (Public domain, so people can perfectly well read it somewhere else. ebooks have basically zero spillover to other pages.)

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4508870 posted 4:38 am on Oct 18, 2012 (gmt 0)


I just think it's ironic that some here will shoot themselves in the foot and then complain at every opportunity how much it hurts.

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4508870 posted 6:30 am on Oct 18, 2012 (gmt 0)

keyplr,
it's the old "beneficial or detrimental choice" for each of us.

Our understanding of those choices has been the longevity of this forum.

To show you how some things change?
You might recall that for the longest while, I refused to personally acknowledge server farms and their IP's because I construed same as them free advertising.
"Today", I've a long thread on Server Farms.

Don

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4508870 posted 7:38 am on Oct 18, 2012 (gmt 0)

Don, I get all that.

What I'm saying is instead of complaining all the time that your images are being stolen or hotlinked, just let the images be used and FB stops at 3. Then you'll get a lot of free traffic; targeted traffic that actually wants to follow that link to your site because of the nice looking snippet & thumbnail.

In fact, that's the only time FB comes to get images - when a FB user posts a link to your web page. FB does not have a web crawler. If you block FB (by UA or IP range) and a FB users posts a link to your site, then the FB image retriever will keep trying to get an image off your web page until it tries every image. It's possible I guess that this may happen for other reasons, although I've never seen it.

Personally, I get a lot of traffic from FB so I have no problem with them getting a couple images now and then. After all, the images are used to send traffic and sales my way. That's the way the web works. You invest something, you get back something. I've found that the ROI is much higher with traffic generated in this manner.

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4508870 posted 10:40 pm on Oct 18, 2012 (gmt 0)

And if you're not getting traffic from this, then you either do not have a page anyone is interested in visiting or you are blocking the human traffic somehow or blocking the image retrieval hence no thumbnail for anyone to click on and get to your page.

If we're going to get insulting, I might observe that you seem to have an awful lot of trouble wrapping your mind around the idea that websites, like people, are all different. Different designs, different goals.

In my specific, individual case: I have never seen a FB referral other than for e-books. And I don't mean the Paston letters; I mean picture books. The ones where the illustration file outweighs the text by a factor of at least ten. These are all dead-end pages. That is, they physically connect to the rest of the site, but human visitors don't continue. They read the specific book they came for, and then go about their business. It's just fluff.

I am much more intrigued by visitors like the Canadian I found interlaced with yesterday's Facebook referrals.* Originally arrived from points unknown to read up on UCAS legacy fonts, but appears to have spent the whole day sporadically returning to assorted "real" pages. Desktop computer with the identical IP as the initial visit's iPhone, which is not something you see every day. Real pages = ones I wrote myself, containing information that people look for. Not a whole lot of people, but when you're on satellite internet you have to be focused.

Now, if anyone has any idea why the Air Force's 754th Electronic Systems Group is interested-- as in, over 40 humanoid visits to date-- in a page devoted to the animal-filled state of an apartment I no longer live in... They don't seem to have shown up in this Forum, but cursory googling suggests that I am not alone in these sudden inexplicable visits. If it's one of those multiple-tabs, reopen-after-crash situations, I can only say that the Air Force needs to upgrade their hardware. They shouldn't be reloading the page several times a day.


* Which, incidentally, managed to arrive without benefit of clickable pictures, so it's nonsense to say you have to let them pick up either three-- on odd-numbered days-- or all-- on even-numbered days-- image files.

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4508870 posted 6:53 pm on Oct 19, 2012 (gmt 0)

If we're going to get insulting

No one has been "insulting."

you seem to have an awful lot of trouble wrapping your mind around the idea that websites, like people, are all different.

I have no trouble with your your web site mission. The irk comes from redundant misinformation about FB. If you don't want their free traffic stream, block them and forget about it.

Bewenched

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 4508870 posted 2:59 pm on Oct 20, 2012 (gmt 0)

I realize that they just have a fetcher, however there is NO WAY that 50 people "liked" one of our pages at the same time.

We've always said that the biggest threat to Google would be facebook. I'm wondering if Facebook isn't developing their own bot.

Regarding them grabbing several images. If you define the image you want them to grab in your meta tags they will only grab one.

<LINK REL="image_src" href="http://www.mysite.com/myimage.jpg">

I didn't get a chance this week to search the logs to see what they were really doing, but the initial snapshot was them hitting pages, not just images.... will try to look further into it this weekend and let every one know.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved