homepage Welcome to WebmasterWorld Guest from 184.72.69.79
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
rarely used
lucy24




msg:4600986
 11:57 pm on Aug 10, 2013 (gmt 0)

Stop the presses.

"rarely used" is the Google image search app.

:: insert boilerplate about found-while-looking-for-something-else ::

Sequence of events:

Do a search using google app on iPad, resulting in SERP including images I recognize

1. Select image from my site (to be exact: image that I replaced over a month ago, meaning that what I see is a cached version-- same as in ordinary browser image search)
2. In a different browser, open a page on my site so that logs will show a clear before-and-after
3. Select "save image" option (which I've never tried before) ... leading to "failed" error message.

Now for logs.

{my IP} - - {time of event #1} "GET /fonts/images/keyboard_explode.png HTTP/1.1" 403 1497 "-" "rarely used"
{my IP} - - {time of event #2} "GET / HTTP/1.1" 200 2732 "-" "{ordinary Camino UA string here}"
{my IP} - - {et cetera with all suporting files}
{my IP} - - {time of event #3} "GET /fonts/images/keyboard_explode.png HTTP/1.1" 403 1497 "-" "rarely used"


:: detour to last few days' raw logs to search for "rarely used" UA ::

Yup. All requests are for images.

Holy ###. Who knew?

 

yaimapitu




msg:4600996
 12:21 am on Aug 11, 2013 (gmt 0)

Just curious: why does your log show {my IP} and not {their IP}?

BTW, I've never seen "rarely used" in a log... :)

lucy24




msg:4601005
 1:35 am on Aug 11, 2013 (gmt 0)

why does your log show {my IP} and not {their IP}?

Because if it showed google's IP, discussion of the UA would have taken a wholly different direction the first time it showed up. As it turns out, the reason investigating the IP didn't lead straight to a Ukrainian server farm is that each one is a perfectly innocent human using a perfectly innocent app.

Further investigation suggests that most mobile users are coming in from google dot com using the device's browser, not the app.

The "rarely used" UA first appeared in May. With the rare exception of people trying to "save" the image, the UA block has no effect on the user experience. But I'll change it to match the /blank.html rewrite, anyway.

yaimapitu




msg:4601019
 5:55 am on Aug 11, 2013 (gmt 0)

Thanks for the info. Add this:
http://productforums.google.com/forum/#!topic/webmasters/meWyH_G84ho
and I think I understand the problem now...

:)

[edited by: phranque at 5:58 am (utc) on Aug 11, 2013]
[edit reason] unlinked url [/edit]

lucy24




msg:4601033
 8:10 am on Aug 11, 2013 (gmt 0)

The final post (dated July 5) is priceless.
it appears to be used by a mobile app that fetches and displays images. maybe it's used by some sort of image-search app?

Yes: GOOGLE'S OWN APP to be exact. The one with the big blue g logo on your iPad. Interesting that nobody stepped in and owned up to it. I only discovered it while checking something about image search on the iPad. When I went to fine-tooth-comb my logs, there it was.

I love the way Google Groups refers back to WebmasterWorld for information-- even if the "information" is nothing more than me noting the existence of the UA!

I originally guessed that "rarely used" is only, ahem, used when the user is not signed in. But I tested and this turns out not to be the case. Apparently the app just isn't all that popular; most people use google within their mobile browser.

wilderness




msg:4601054
 12:15 pm on Aug 11, 2013 (gmt 0)

I love the way Google Groups refers back to WebmasterWorld


lucy,
FWIW, Usenet (i. e., newsgroups of which there are tens of thousands, maybe hundreds) use to come standard with most of the internet providers. I participated in UsenetNewsgroups of specific topics for more than a decade (The webmaster forum use to be superb).

Most major internet providers have stopped providing the service. It may be purchased from other providers.
In any event, Google Groups was developed because of the decline in access of Usenet. It's a POOR substitute.

Don

not2easy




msg:4601100
 5:30 pm on Aug 11, 2013 (gmt 0)

Thank you for this little tidbit, lucy24. There is such a small number of victims I can go back and unblock them...unless it was the IP that got them blocked in the first place.

lucy24




msg:4601131
 8:54 pm on Aug 11, 2013 (gmt 0)

You may want to do what I do, both here and with /blank.html referer (also image search): rewrite to a one-pixel gif. It has no effect on what the human user sees, and it uses less resources than any other action you can take.

Google Groups was developed because of the decline in access of Usenet

I would never have seen the GG reference at all if yaimapitu
:: detour to look up the word* ::
hadn't pointed it out. Personally I gave up on GG a couple years ago when I was trying to find the current discussion threads on "Google in your language" (user-generated translations of the search pages). No matter how many times they redirected or pointed to newer formats or asked for a fresh login, I could never find any discussion on this topic more recent than 2008.


* "People from Yaeyama islands", it says here. I thought yama was mountain, but, well, whatever.

yaimapitu




msg:4601164
 1:43 am on Aug 12, 2013 (gmt 0)

Googlegroups (or the similar Yahoogrups) may not be as good as usenet news (used to be), but they are popular with the poltical/social activist crowd, it seems. The NSA is probably getting a full feed. LOL

OT, re the footnote:
I thought yama was mountain

You're right. The island in question - a delightful place in the middle of nowhere - has lots of them yamas (I spent a few days there earlier this year, in connection with a website project, and that's how I learned that word I'm using as ID here). :)

loupiote




msg:4626989
 5:46 am on Dec 1, 2013 (gmt 0)

thanks guys for confirming my suspicion.

i am the author of the google thread that you are referring to :)

i wonder if google has been made aware of this "rarely used" user agent that their apple app use and that is (obviously) not listed here:

https://support.google.com/webmasters/answer/1061943?hl=en

it is really not cool that an app developed by google is using such a non-informative user-agent. that goes pretty-much against google policies regarding transparency.

keyplyr




msg:4627003
 8:26 am on Dec 1, 2013 (gmt 0)

While I have no knowledge of a Google app using the "rarely used" UA, I have blocked several server farm IP ranges using it, and yes they were after image files as I recall.

phranque




msg:4627012
 9:22 am on Dec 1, 2013 (gmt 0)

welcome to WebmasterWorld, loupiote!

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved