Google favicon

[keyplyr]

First I've seen of this so apologies if this UA has been around awhile. Coming from various valid Google IPs:

UA: Mozilla/5.0 (Windows NT 6.1; rv:6.0) Gecko/20110814 Firefox/6.0 Google favicon

Requests page, the the favicon.ico. It was very busy on my server.

[Pfui]

I just stopped by to mention the same sighting and you beat me to it. (smiles) Yep, the old UA was the same old, longer UA, and since yesterday it's just "Google favicon".

But the conduct's different. It only hit root, and did NOT ask for favicon.ico (or favicon.gif). Go figure.

On a related note -- and one that set my hair on fire at the time and changed what and how I communicate with gmail accountholders forevermore --

Last Jan. 19th, I sent a private, path-unpublished, billing-related link (to root in a private directory) to a client's assistant at his gmail account. He replied on Jan 21st. A week later, on Jan 27th, the old "Google favicon"-ending UA made a beeline for that private directory. It took the whole tax data-rich .html file -- dammit! -- and also favicon.ico.

I've long heard rumors Google harvests private data it 'finds' via gmail links but here was very disturbing proof. And it used a misleading UA to boot.

[keyplyr]

I've long heard rumors Google harvests private data it 'finds' via gmail

Ahhh, but in internet terms, it isn't private if it lies in a public directory. Google could not access the file if it did not have your permission. Publishing a file in a public directory is essentially giving permission to the public, including Google.

If you want it private, use a password protected (private) directory.

[lucy24]

Requests page, then the favicon.ico.

... and, unlike certain other faviconbots one could name, it will request the favicon even if it received a 403 when requesting the front page (as happened, predictably, with the old faviconbot that came with no UA at all). I suppose the point of requesting the front page first is to verify that the host exists at all, as opposed to redirecting to some other hostname.

[dstiles]

Unfortunately several of my clients use their mail service and I HATE sending secure details such as passwords and usernames to them because I simply do not trust them! Some clients even want mail to their domains delivered direct to gmail, despite my warnings.

The sad and annying thing is: G is now everywhere. It is almost impossible to perform any online transaction from error reporting to payment without involving them in some capacity. I complained to a company only a few days ago about an unreadable CAPTCHA on their support forum and the fact it was G's, who would most likely have logged my access details; I've complained to my domain registrar about having to accept G's javascript - often several (though I'll never permit their analyzer); and so it goes on.

They even sent my limited company a letter urging me to join some online group or other of theirs - immediately shredded!

[keyplyr]

I wouldn't take it any more!

[Pfui]

Ahhh, but in internet terms, it isn't private if it lies in a public directory.

As I stated x2, the directory was private. Oh, and the directory it was in was also private. In addition, the directories don't exist in the site's robots.txt, and they're not guessable. Thus the likelihood of Google finding the invisible, double-private data without crawling the link sent in private e-mail is... nil.

Google could not access the file if it did not have your permission.

Well, that's baloney. Google did not have my permission, neither explicitly nor implicitly. Or are you trying to be flip about a serious event? Not amused, sorry.

If you want it private, use a password protected (private) directory.

Sure. And send the access data to the assistant via -- wait for it -- their gmail account? Riiight.

Better to screw things down tight via .htaccess and block everything remotely related to G even in clearly private directories. Now the assistant can access the link without any unpermitted e-spy secretly, and repeatedly, tracing his tracks.

[keyplyr]

As I stated x2, the directory was private. Oh, and the directory it was in was also private

Did you give Google a password? How did it get in if the protected directory is set up properly? I mean really...

Don't intent to be antagonistic, but I hear this stuff a lot from my clients. They may consider a web property private, but unless it is set up to be so, it isn't.

[Pfui]

How did it get in if the protected directory is set up properly?

You're conflating "private" and "protected" and arguing that if all directories/files aren't the latter via htpasswd, it's okay that Google -- or anyone? -- examines the former. Really?

Imagine you have a rarely used, small wire snake you keep stashed in an old box under your bathroom sink at home. A beard-shearing guest that's staying over texts you at work, says the drain's backed up. You e-mail back where to find the snake. Guest finds, fixes, finis.

A week later, shorn guest gone, you come home and discover someone's been inside, dug under your bathroom sink, opened that old box, and examined the snake. That's called trespassing. If they took the snake, that's called stealing.

Basically Google did both to my hidden and private file and had no more right to than, say, China, or Amazon AWS, or the Xfinity WiFi jockey down the street.

If you disagree, c'est la vie.

[keyplyr]

No I'm not conflating the two. There is no such thing as private when you publish anything on the world wide internet. The only way to stop unwanted access is to restrict it.

And I'm not arguing that it is OK for Google or anyone to do anything. I'm just stating the facts. Ethics aside, whether you like it or not, if it is accessible, it will be found. Whether you call it private or not, it will be found. To make it *not* be found, restrict access by one of the standards set for that purpose.

One of my clients got upset because the photo of his cat ended up in the Yahoo image search. He thought it was private because the photo had never been posted anywhere, neither had the directory where the file resided. However he used a browser to view this photo at least once - voila! The photo had now been requested from the server and the server had fulfilled the request to the public internet. That's enough.

How did Yahoo pick-up on this? How did Yahoo learn where this file resided? Who knows, maybe through an add-on to his browser, but the fact was it was publicly available. The file was in a public directory, available to anyone who knew where it was.

[tangor]

No I'm not conflating the two. There is no such thing as private when you publish anything on the world wide internet.

Splain that, kiddo. is gmail email accounts "the world wide web?"

Get a pair of glasses, please, as this is a fairly serious incident.

Then again, I agree that private (as in unpublished) and protected (as in password) are tow different things.

The biggie here is G getting the link from an email on their email server!

Not a G basher, or apologist, but merely a fellow who will never give this company an inch as they always want a mile, and will do whatever it takes to get that and 100 more.

[keyplyr]

You get "Get a pair of glasses." The files were on a web site server. The gmail reference was about conveying that info to the other person... talk about conflating.

However, has anyone read their ToS or privacy policy at gmail? When you signed up for a gmail account you gave Google permission to access all email files. The content of those emails may be "private" excluding others, but your host (Google) was given permission by you.

However, I'm not a Google fanboy by any means. I think they have gained too much control over the web. But I'm also a realist. You give up a lot of privacy when you publish. It comes with the territory; notice I said "you give up." Stop bitchin' and complaining about Google. You gave them the power after all.

[tangor]

I rest my case. :)

Iffn you can't see it, thee rest can, and it that don't make a bolt for the door, I suppose no much will.

the majority think that email is private.

And that's the point.

Oh.... gmail is not a web public site. Just ask google.

[keyplyr]

it that don't make a bolt for the door, I suppose no much will

I'll have to check with my linguist and get back to you.

[toidi]

The way i see it, the sender did not agree to the gmail tos, so goog was trespassing when they went in the private directory. Not much different than my backyard, it is out there in public for all to see but it is private and anyone who wanders there without permission is trespassing and subject to applicable laws.

[keyplyr]

Which is likely why, at any given time, Google is involved in a half-dozen law suits. Internet privacy is still being negotiated, with many countries interpreting it much differently than others.

[Pfui]

tangor & toidi: You got it right. In this case, Google was/is wrong. Intrusive, invasive, and wrong.

And as dstiles illustrated up above, Google is everywhere. My husband uses an Android phone, and G has records of his calls AND my calls to him. If I leave v-mail, they have that, too. (A few years ago, G publicly released searchable private transcripts -- oops -- and people rightly freaked out on so many levels.) [pcworld.com...]

One more: An annual, four-day civic event in Seattle in August requires neighborhoods in its "high impact areas" to obtain street-access passes to pass blockades and reach their homes. First and last names, physical addresses, e-addresses, phone numbers, etc. This year, the new, mandatory sign-up method to get home? Google Docs.

So much for privacy.

So anyway, getting back to the OP...

Google UAs either wholly named "Google favicon" or carrying the phrase as a suffix do much, MUCH more than merely gather favicons (if they even do that).

[Pfui]

P.S. Just one more privacy-, home-, and life-related G tangent, please.

Dropcam (wireless video monitoring cameras & 'video history' storage) got a rave review from the Federal Way (nr. Seattle) Police Department for quality footage of a home break-in. [kirotv.com...]

I looked into them and discovered Dropcam was owned by Nest (home automation products, incl. thermostats & CO detectors & security). I looked into them and discovered Nest was owned by -- Google.

Mind = Blown

[tangor]

it that don't make a bolt for the door, I suppose no much will

I'll have to check with my linguist and get back to you.

No linguist required. Typo checker would be more appropriate. Make no apologies other than going blind (in real life) and sometimes I just don't catch the fat fingers syndrome before hitting Submit..

[lucy24]

the sender did not agree to the gmail tos

Sender may or may not have agreed, but recipient definitely didn't-- unless the reported behavior only occurs when sender and recipient are both gmail addresses. The analogy that presents itself is consent to a recording. In the US, the Great Divide is between one-party and two-party consent.

If you put your party invitations on a postcard, that doesn't mean you've invited the mail carrier.

[Pfui]

Quick Recap:

I was the sender using a private, non-gmail account. I sent a private e-mail -- containing a private link to a private file on a privately-owned server -- to a private gmail account. Ironically, the file path even included the word "private".

Within a week, Google used the longer-name UA detailed in the OP to crawl that private link from:

google-proxy-66-249-84-140.google.com [66.249.84.140]

Approx. two weeks later, Google used the same UA to again crawl that private link, this time from:

google-proxy-66-249-84-148.google.com [66.249.84.148]

Kind of like in WAR GAMES when Joshua called back not once but twice.

[wilderness]

And as dstiles illustrated up above, Google is everywhere. My husband uses an Android phone, and G has records of his calls AND my calls to him. If I leave v-mail, they have that, too. (A few years ago, G publicly released searchable private transcripts -- oops -- and people rightly freaked out on so many levels.)

When I first began using g-mail, immediately (and regularly; every week or less) I began deleting my sent and trash copies (use an email software rather than web access), pop-ups began appearing notifying me as I deleting the old items that g-mail automatically deletes old stuff after x-days. This showed me the capabilities and intentions of Google/G-mail.

Some years ago, a contact requested that I add other family members (when sensing relevant materials of their family patriarch) via CC.
After a time, my internet provider (ATT) notified me that a spam complaint had been filed.
They offered an explanation/excerpt and used the term 'as an example'. Searched my old stuff and provided the subject line, which ATT verified was the complaint source.
ATT also informed me that technically speaking, if a 3rd party forwards one of my emails without my knowledge, than under normal email TOS that is also spam by myself.
As a result, it was suggested to add a disclaimer to my emails.
FWIW, I stopped sending materials to the family, despite request to continue. All the complaint had to do was request removal and it would have been done in minutes.

[keyplyr]

Well if all that privacy dereliction of your emails has contributed in any way to the nice fat checks I get every month from Google, I'd like to take this opportunity to thank everyone here and just let me add that at no time have I personally read any of your emails, at least not to my recollection.

[tangor]

Where's the Don't Like or Vote Negative button?

[keyplyr]

Where's the Don't Like or Vote Negative button?

Sorry, I must have worn it out on your posts :)

[tangor]

Glad you got the last word. Whew! Afraid it might have been me!

[keyplyr]

If that's how you see it :)

[tangor]

Wow! You just don't give up do you? You've already saved Google and their indiscretions and your earnings...

[keyplyr]

tangor if you are so motivated, why don't you just send me a stickymail

[lucy24]

And returning to the original subject, which saves me the bother of deciding whom to put my money on ...

There it is, by gum:

66.249.81.195 - - [25/Jul/2015:10:26:37 -0700] "GET / HTTP/1.1" 200 2694 "-" "Google favicon" 
66.249.81.189 - - [25/Jul/2015:10:26:37 -0700] "GET /favicon.ico HTTP/1.1" 200 1751 "-" "Google favicon" 

Is it too much to ask that they pick one UA and stick with it?

:: wandering off to edit "ignore" section of log-wrangling script ::