homepage Welcome to WebmasterWorld Guest from 54.237.213.31
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
New google IP range
173.194/16
dstiles

WebmasterWorld Senior Member dstiles us a WebmasterWorld Top Contributor of All Time 5+ Year Member



 
Msg#: 4455681 posted 9:27 pm on May 20, 2012 (gmt 0)

New (to me) google IP range at 173.194/16

First (known) hit today was rejected when a query came in from Brazil using an IP in this range as a "translate" proxy.

I found no mention of googlebot in the DNS for 173.194/16 so complete range blocked.

 

Staffa

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 4455681 posted 10:23 am on May 21, 2012 (gmt 0)

Thanks for the heads up, duly noted :o)

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4455681 posted 4:17 pm on May 21, 2012 (gmt 0)

Can't be brand-new. I've got it flagged as "google.co.uk". Maybe I should start putting a date on this stuff. Saves detour to raw logs

:: detour here ::

Oh, that's odd. I've got the exact IP 173.194.99.20 and it isn't a British query at all. Both are Google Translate-- one without referer, the other citing translate.google.com.mx (The requests themselves are unimpeachable; it's my most common translation by far.)

:: further detour to whois ::

Huh. Wonder where I got google.co.uk from? Now it just says google.

Funny you should mention Brazil, because only yesterday (really) I was visited by something calling itself the googlebot. Didn't bother to look up the exact IP, but it came from LACNIC territory. Truism: A Ukrainian robot doesn't necessarily have to come from the Ukraine.

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4455681 posted 7:04 pm on May 21, 2012 (gmt 0)

IMO - Can't imagine why any site admin would allow translate tools. They disable browser side scripting removing ads, they display your content from their servers allowing scraping, you loose stats, etc. This is a huge security hole.

dstiles

WebmasterWorld Senior Member dstiles us a WebmasterWorld Top Contributor of All Time 5+ Year Member



 
Msg#: 4455681 posted 9:08 pm on May 21, 2012 (gmt 0)

On the other hand, one of my customers gets orders world-wide and says he needs the translate feature - not that many translates get through because the IPs have already been blocked by some other stupid G app sharing the same IP ranges.

Lucy - the DNS entry is old-ish (2009), which surprised me, but I've seen nothing "bad" from the range before or I would have logged it.

As a range, it's sandwiched between a softlayer /15 and reliablehosting. Since DNS was updated Feb this year I wonder if they've recently "borrowed" it from around there hoping to bamboozle us. :)

Staffa

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 4455681 posted 10:18 pm on May 21, 2012 (gmt 0)

I wonder if they've recently "borrowed" it from around there hoping to bamboozle us.

Most likely.
I also work with a GeoIP dbase and most often the ranges that are worth blocking are sandwiched that way.

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4455681 posted 11:12 pm on May 21, 2012 (gmt 0)

Can't imagine why any site admin would allow translate tools.

Uhm... Because the site contains information that doesn't happen to be available in Hungarian? (I made that up.)

This is me you're talking to. If I had advertising it would net me about seven dollars a year. And the only reason Translate doesn't get into stats is that I've physically blocked it (see unrelated threads about google peripherals acting on javascript), along with Preview and similar.

Google-Mexico is utterly expected, because this particular page is the English translation of a book originally written in Spanish. The Spanish version exists online, but is not as pretty. In fact I've added lines to the hotlink routine to let people see the pictures. Oh, and it's public domain, which means it's already been scraped.

And if someone wants to read my original content in Russian, more power to 'em ;)

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4455681 posted 7:40 pm on Oct 10, 2012 (gmt 0)

If you allow translate tools then why bother with any server security at all? Translate tools scrape your entire site's content and put it on their servers where it's a free for all.

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4455681 posted 10:32 pm on Oct 10, 2012 (gmt 0)

How'd this thread happen to get bumped?

Don't know about anyone else, but in my case, 99% of the translation requests are g### translate putting a specific page into Spanish-- meaning that an 1894 text loosely translated from Spanish into English around 1911 is being machine-translated back into Spanish :) The numbers are probably lower now that I've put up a Spanish version of the same page. Using the original (public domain) text, not a retro-translation.

And I don't have any content that anyone would benefit from scraping. Trust me on this. (Or snoop. Most people do, sooner or later.)

ymmv

Bewenched

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 4455681 posted 1:13 am on Oct 26, 2012 (gmt 0)

173.194.73.106

is one that I got a visit from. In my tool the domain comes up as 1e100.net via geoip which is
    hosted by google
.

Discovered Nameservers
ns1.google.com | 216.239.32.10
ns4.google.com | 216.239.38.10
ns3.google.com | 216.239.36.10
ns2.google.com | 216.239.34.10

Registrar Information
Markmonitor.com

Man these guys are getting smart having google host them. How the heck are we supposed to stop them now. grrr...

g1smd

WebmasterWorld Senior Member g1smd us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 4455681 posted 8:23 am on Oct 26, 2012 (gmt 0)

In mathematics, 1e100 is a googol.

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4455681 posted 8:39 am on Oct 26, 2012 (gmt 0)

Or, for us ordinary mortals, 10^100. Is that the number of IP ranges g### controls?

:: enjoying rare thrill of using literal ^ without having to \ escape it ::

dstiles

WebmasterWorld Senior Member dstiles us a WebmasterWorld Top Contributor of All Time 5+ Year Member



 
Msg#: 4455681 posted 9:01 pm on Oct 26, 2012 (gmt 0)

Bewenched - block the complete range 173.194.0.0/16 - as noted in the OP, I saw no (valid) bots there.

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4455681 posted 12:20 pm on Nov 4, 2012 (gmt 0)

I've edited out the majority of the refer info.

62.20.191.zzz - - [04/Nov/2012:11:38:05 +0000] "GET /MyFolder/MySub/MyPage.html HTTP/1.1" 403 559 "http: //www.google.se/url?" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; GTB6.3; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; InfoPath.1; OfficeLiveConnector.1.3; OfficeLivePatch.0.0; BRI/2)"
173.194.98.16 - - [04/Nov/2012:11:38:10 +0000] "GET /SameFolder/SameSub/SamePage.html HTTP/1.1" 403 559 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; GTB6.3; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; InfoPath.1; OfficeLiveConnector.1.3; OfficeLivePatch.0.0; BRI/2),gzip(gfe)"

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4455681 posted 12:13 am on Nov 5, 2012 (gmt 0)

You don't let Swedes buy your widgets? You are restrictive aren't you :)

Interesting sequence. Can g### tell when clicking on a search result leads to a 403? I hope they don't think there's a Cloaking issue :(

I've been blocking the plainclothes MSNbot for ages. Is it time to block plainclothes googlebots as well? Or would that be counterproductive?

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4455681 posted 12:32 am on Nov 5, 2012 (gmt 0)

lucy,
I've some longtime correspondents in Sweden, Denmark, Finland and Norway that I make exceptions for. The problem with the EURO IP assignments is that the procedures are vastly different than their NA counterparts as to the definition and application of "dynamic IP's".

It's a real PITA to break a class A down to a small Class D range and generally involves four lines, and then have it change to either a different Class A or B within a few hours.

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4455681 posted 2:54 am on Nov 5, 2012 (gmt 0)


I get visitors from Sweden, Denmark, Finland and Norway, also Belarus, Ukraine, Russia, Lativa, Vilnius and Estonia. They even buy stuff occasionally. I remember doing the math with conversions several times :)

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4455681 posted 3:22 am on Nov 5, 2012 (gmt 0)

The problem with the EURO IP assignments is that the procedures are vastly different than their NA counterparts as to the definition and application of "dynamic IP's".

Not even unique to Europe :( My IP has bits of, I think, 67, 69 and 71. So if there is a power outage, or I deliberately turn off the modem, I am likely to find myself in an entirely different A range. And I've currently got a Canadian specimen of those users who open a page in a back tab, forget all about it and therefore reload every time they open the browser. So far they've stayed in the same A, but last time I looked, they'd hippity-hopped to a whole new B. Non-contiguous, at that. The intervening B's are in a different part of the province.

Now, if you had a version of your widgets that worked with sled dogs, you'd be sitting pretty. Some of those satellite addresses go right down to /30 and stay there. Not like the southern version that can change IP in the middle of a download.

It's a real PITA to break a class A down to a small Class D range and generally involves four lines, and then have it change to either a different Class A or B within a few hours.

Can't you use cookies instead? Let them in once, set the cookie, and they can override the IP block forever.

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4455681 posted 1:03 pm on Nov 5, 2012 (gmt 0)

Can't you use cookies instead?


lucy,
I'm sure that's a possibility.
Unfortunately, I'm not looking for new mountains to climb (or even comprehend), rather at this point, I'm more interested in avoiding jumping over cracks in the sidewalk ;)

I have another 3-4 years into a sixteen-year project and after that it wouldn't bother me if the internet and everything related to networks dropped of the face of the earth.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved