Location by IP

I know the answer, but I may be missing something

[rocknbil]

Geolocation by IP is 50-85% accurate, depending. I have a client that is opening an office in Hawaii and has it set it in their mind that they can automagically determine the Hawaii location by IP, and the project is moving forward like a locomotive based on this straw holding up the camel's back, in spite of my whining that it is bereft with problems. I've been in discussions with MaxMind and a couple other geolocation services and while they all want someone's wallet to crack open, the bottom line is "Hey man, if the server is in New Jersey, that's where it is!"

Let's not even get into Satellite providers, which I imagine (not "know") are a are portion of Hawaii's bandwidth.

The locomotive arrives at the station and is ready to test. First hit on the site from Hawaii, "it doesn't work." Of course not. They are dialing in from a company who's data center is in TEXAS.

They tried another location. KANSAS, on a RoadRunner backbone.

I've suggested the only reliable method would be via user input, just enter a zip code and that is all, but this is "unacceptable," as if some magical solution will just present itself.

However, being one that knows there's a truckload I don't know, maybe I'm wrong. My question is, am I missing something here? Is there some reliable method to determine a Hawaii location by IP?

[g1smd]

No.

[Staffa]

I have been working with Maxmind's database for a number of years and for my purpose it works very well.

However, there are no specific ranges determined for Hawaii, that state comes under general USA.

So unless your client has someone sitting, twiddling their thumbs who could manually clean up the database (one or two weeks work) your client may have to abandon his idea.

[piatkow]

"Hey man, if the server is in New Jersey, that's where it is!"

Even if the user is in a different country, I have encountered one international organisation who routed all their internet traffic through a server in New York. You couldn't tell which continent the visitor was on, let alone pin down a location within a country.

[brotherhood of LAN]

Yeah there's a ton of those kind of services that hide the 'real' location of someone.

Perhaps there's some market research regarding the competition of internet providers in Hawaii that could indicate how many people are on landline connections, satellite etc.

Some of the IP-location lists use triangulation to determine someones location, basically you measure the latency to connect to the IP from various locations which can indicate where they are based on the time differences between the requests. It's not something I've ever tried and using a zip code sounds much more sensible.

[rocknbil]

Thank you everyone for your input.

No.

LOL . . . how did I know it would be g1smd with that one. Thank you. :-)

there are no specific ranges determined for Hawaii,

Eh? Even with the light database, there's tons of IP's listed in Hawaii. The problem is, there's tons more users using bandwidth with IP's not located in Hawaii. :-)

manually clean up the database

I'd even be willing to take that impossible to maintain project on - but the problem is that those IP's are not restricted only to Hawaii. There's a reason their data center shows up in Texas - one scenario is that it's indeed a satellite, in which case thousands of mainland users would be on the same IP. Identify it as Hawaii . . . and poof, they all get zapped to the Hawaii site. No deal.

there's a ton of those kind of services that hide the 'real' location of someone.

Sometimes it's not hiding. I used to be on satellite in rural Oregon, and most of the time the geotargeted ads placed me in New Jersey. :-) When our satellite malfunctioned, it would report me as Houston, Texas.

Some of the IP-location lists use triangulation to determine someones location, basically you measure the latency to connect to the IP from various locations which can indicate where they are based on the time differences between the requests.

Fascinating! Have you got names?

I've been in communication with Maxmind, IP2Location, Geobytes, a couple others that didn't seem very promising, and I had a strong look at what Skyhook is doing (which sounds similar to your idea,) but it sounds like Skyhook's main product is an SDK to create apps on mobile devices, which seemed to mean geolocation of the device in your hand, not by IP. Abandoned.

[dcheney]

Since Hawaii is in a relatively unpopulated timezone, perhaps you can use that (for example, JavaScript function getTimezoneOffset())

[Demaestro]

Here is an insane idea.

Seeing as Hawaii has it's own timezone....and assuming people's computers are set to such... Using javascript it should be possible to detect the time and see if it is within 10 minutes of the time in Hawaii or maybe it is possible to determine the timezone from the user's clock directly?

Once/If you have that info you could aJax it back to the server, set a cookie or session variable or something.

I am running on low sleep though so I am not sure how feasible this is

[rocknbil]

That's actually quite brilliant. :-)

I've never been hip on making anything Javascript dependent, and it would require a double hit on the server for every request, but it could work . . .

[lucy24]

Two minds with but a single thought? :) (Note two adjacent timestamps.)

I was thinking the client could restrict his service to a narrow pool of users: students living in campus housing, and people occupying those overpriced housing complexes that come with everything pre-installed including Internet. That puts you in the rare and extraordinary position of being able to derive someone's street address from their IP address.

Not so useful of course if he's trying to target the entire state of Hawaii including outlying islands.

Time zone is really not a bad idea if you don't mind Polynesia and the Cook Islands coming along for the ride. And those are other countries, so you should be able to exclude them easily.

[Leosghost]

That's actually quite brilliant. :-)

I agree completely :) ..this thread and the problem has been drawing me in a few times today..

dcheney and demaestro..have virtual cigar / beer / wine on me ..lucy24 too ..and rocknbil..in fact every one in the thread ..intriguing thread..:)

[brotherhood of LAN]

> JS time

Agreed that's a very eloquent idea to use, the fact that Hawaii has its own timezone is very handy to know here!

>Have you got names?

I don't, but I'm aware that some use the technique. Google must. When I'm in Canada the best most IP->location services can do is list where my ISP is (30 miles off) and can be found by a simple WHOIS lookup anyway (which is worth mentioning as a solution on its own but IMO isnt the best one in the thread now).

Google can nail me down to within a mile usually. Outside of WHOIS info, I would assume theyre getting the better granularity by using the technique.

[g1smd]

Google can also triangulate you from the wi-fi routers you can get signal from having mapped many of their locations. They also continue to get new data from mobile phones that send back their GPS co-ordinates with requests and these can also be associated with router names and/or IPs.

[Leosghost]

I have a maxmind account,( paid, the "live service", not the downloadable DB ) and their service pins me down to within 600 to 800 yds of my house..if I go to one of my sites which I have the script running on that talks to their servers..
G is not so good ( when I allow it to track me sometimes to see how it is doing ) it frequently puts me in the next town over, about 2.5 miles away..( it seems to have a problem putting me on the correct side of our small town population 2000, inspite of the fact that our town has its very own dslam ), unless of course the plex is toying with me :)

[Demaestro]

Wow I can't believe someone else was typing the timezone idea at the same time as me... crazy

It is neat that the fact Hawaii is so remote was the cause of the issue and the same fact also provided a possible solution.

I am curious to see how this one plays out. I still can't believe all this over the client's desire to not have users self identify.

[rocknbil]

Heh. Don't work with marketer teams much huh? :-) <At times it feels like> they pretty much assume the advice from developers doesn't really matter and we are just being difficult, then forge on in spite of realities. Some Important Person at some Expensive Seminar told them this could be done, so it surely must be true. So much, in fact, that even though the first attempt isn't working out, they're applying it to a second site without hesitation. ("Insanity is doing the same thing over and over again expecting different results" - paraphr.:Einstein)

I've discussed the Javascript time zone approach and am on the fence about it due to the fact a page must entirely load before it redirects, if it has to. Seems cheesy. :-\ If I have to use Javascript, I'd rather use it to pop up/lightbox a zip code entry so the user has some clue as to what's happening.

We'll see what they say.

[swa66]

I'm using the for-pay maxmind database on an IP to country level: it rocks.

Their free ones: they had enough errors bad enough to not want me to continue to use them, but once I tried the for-pay one: I've immediately signed up for autorenewals. I've so far found one error and they corrected it in a matter of days.

Integration of it is simple enough, and it's fast.

Their apache module makes it really easy and gives it in the environment and makes it usable in e.g. rewrite rules.

They have a more detailed IP to city level in addition to the country level one (which I'm using).

[incrediBILL]

Is there some reliable method to determine a Hawaii location by IP?

I'll ditto G1's answer: NO!

You can get close but no cigar.

The timezone suggestion is actually pretty good for Hawaii, nice to see people thinking outside the box.

What's more likely used even by Maxmind to narrow down the results to Hawaii is the whois records and reverse DNS results which would most likely be used to narrow down local service providers.

For instance, services like Comcast, Road Runner, etc. typically has the location coded in the reverse DNS of the IP address.

I've discussed the Javascript time zone approach and am on the fence about it due to the fact a page must entirely load before it redirects, if it has to

Put the JS in the <head> of the page in the so it should execute before the rest of the page displays. You only have to do this once before you tag it in the session as a Hawaiian timezone so it's really not so bad IMO.

I'd also back it up with server side GeoIP testing so if you can accurately detect it in some cases server side you get better results.

Also, consider building your own GeoIP table using the results of the time zone detection javascript to improve your accuracy over time.

Once you get a big enough sampling, maybe Maxmind or someone will be interested in buying your Hawaiian GeoIP data from YOU! ;)

[Leosghost]

I spoke at length with Maxmind when I first began using them ..They told me that their data base ( particularly at city/ town level ) was constantly updated when the new users of their system logged in..So as I was the first to use their system from my small Brittany town, they then knew that my fixed IP was corresponding to that map location and could use it , as they use all of their customers locations, to fine tune their system..

Also some of the French ISPs ( mine is one of them ) add a 3 letter code to the dslam and the end customer IP ( mine in this case ) ..so the complete string helps them narrow it down even more ..it gives out the town code , the unique dslam number , the "department" ( corresponds to "county" in the UK ) number, and finally the IP number..

Testing the script I use for cities and towns with their data base using their "live service" , and contacts in other towns around the world tends to lead me to believe that many other ISPs as well as those mentioned by IncrediBill add these "extras"in the reverse DNS..

The one thing that might throw you off a little is AOL customers..Maxmind did warn that in Western Europe for example, all AOL customers in France, the UK and Germany, and maybe other countries appear as UK IPs due to AOL procedures..

Google apparently does not use a system as accurate, as adsense frequently serves me German ads..

[lucy24]

You mean you're not really in Paris?! Does Bécassine know?

When the big print says Yellowknife and the little print says kugluktuk-hyphen-stuff-hyphen-morestuff, you're in "Who am I gonna believe-- you or my lyin' eyes?" territory.

My own IP is less helpful. The big print says I am in San Francisco and the little print says I'm in a place called ADSL. Yup. And I'm not certain that the entire population of Canada lives in Ottawa.

[bluntforce]

Apologies as I haven't read through the entire thread.
Hawaii and Alaska share timezones I believe.
Alaska utilizes daylight savings time which Hawaii does not.

If the server is in New Jersey, then there will be a bit of lag.

Cultural differences are perhaps a bigger barrier than IP location issues.

[lucy24]

No, Alaska is an hour later than Hawaii-- two in the summer, thanks to DST. Detour to the globe tells me that much of Alaska spends its life on the equivalent of triple DST, because it's essentially running on Juneau time.

[Solution1]

Perhaps you could use the HTTP_ACCEPT_LANGUAGE header that the browser sends, with the preferred languages, to improve on the accuray of the location.

Obviously, when Hawaiian is among the preferred languages, the visitor is from Hawaii. When Polynesian languages, or Cook Islands Maori, are in this header, the visitor is not from Hawaii.

[g1smd]

And that begs another question, do you want to detect people who are *in* hawaii or do you also need those who are *from* Hawaii but presently located elsewhere?

[incrediBILL]

Obviously, when Hawaiian is among the preferred languages

That would be quite impossible since AFAIK Hawaiian isn't even on the list of languages browsers support.

[lucy24]

AFAIK Hawaiian isn't even on the list of languages browsers support

Safari will use your system languages in order of preference, even if neither the OS nor the browser itself has a user interface in the language. That's why some sites think I'm Canadian when I visit using Safari.

:: quick detour to Language & Text section of System Prefs ::

Yup, there is is, spelled 'olelo Hawai'i (I've omitted one macron and simplified the glottal stops), alphabetized between Hrvatski (?!) and Ido (double ?).

But the issue is a bit academic since I'm tolerably sure that much more than 50% of the population of Hawaii are haoles who don't know a word of anything but English. I know someone I could ask.

[Leosghost]

Not forgetting a substantial number of Japanese etc whose machines ( if bought outside Hawaii ) would be set to their own native languages ( and some of those using machines bought inside Hawaii would have set their machines to their own mother tongues if available on the particular version and OS ), using language settings would probably introduce red herrings, or what ever the Hawaiian equivalent is ..

[swa66]

First I'd see why they need it. If they can live with a certain % of inaccurate results or "don't know" cases and deduce from there what strategy and what certainty level I'd want.
I'll assume it's a sales relating thing - not a legal requirement or something that's secret somehow or so.

I'd use 3 modes to determine what I do:
- know mode: you know what it should be
--- you still offer a "change" -> track it - if it's used a lot you did not know after all.
- doubt mode: you have a guess at best: up to the custoemr what to do, but
--- make sure to make the "change" stand out and track it heavily: it might get the IP reclassified if there's a consistent answer from the users (confirming your suspicion as good enough), or it might show there's no way to know
- ask mode: you know you do not know or you know the users tell you inconsistent answers of what it should be, or you know it's politically incorrect to assume -> ask the user.

If a user resets your choice:
- know mode: just record them and review if you get many changes
- doubt mode: quite interesting to try to change the mode
- ask mode: just record them and review if you get systematic answers
Always: store it in a cookie for that user so they get the view they want.


As input in your choice, use a combination:
- detect problems like proxies, -> try to "see through it" by figuring out the real IP (javascript/headers/...)
- a for-pay database like Maxmind's city one -> it gives the location of an IP pretty reliable if it's not from "funny" ISPs. [Maxmind city one gives longitude and lattitude -> islands make that easy to box it in - no need to ahve a list of cities/zip codes etc.]
- timezone and/or langauge settings in the browser can help too.
- and obviously a reclassify database based on you reclassification answers.


That's the best you'll get it I think.



I'd have a look at how apple does things:
www.apple.com: US shop
www.apple.nl: Dutch shop
www.apple.be: asks the language (too politically loaded to set it yourself), gives Belgian shop in chosen language.

It's not state based, but it shows maybe that if they choose to now always try to detect it, there might be a good reason for smaller companies not to make the choice for the visitor either - or at least not in all cases.

[topcoder]

It's not possible to be 100% accurate. Just recently, I took a trip on Delta Airlines in the US. They have WIFI enabled on some of the planes now, it's actually the first time I've been on a plane with Semi Free WIFI.

Anyway, while I surfing on the WIFI and while the plane was travelling 550 miles per hour, my IP that I was assigned stayed the same through six different states.

I checked it many times through the trip using unlocktheinbox.com, never changed in over 2000 miles.

The time code seems like a much more solid predictor, unless the person is travelling with a laptop that doesn't adjust time. Most newer cell phones adjust time now based on GPS locations.

But I agree the best way is just to ask the user.

[incrediBILL]

Safari will use your system languages in order of preference, even if neither the OS nor the browser itself has a user interface in the language.

While there is an entry in the ISO-639-2_utf-8.txt [loc.gov] for "haw|||Hawaiian|hawaïen", I've never ever seen it in actual usage for accept-language but someone else may have.

MS even dumped the Hawaiian font in Windows 7 that used to be in XP and now you have to get it from a 3rd party source.

I'd have a look at how apple does things:

We're talking Hawaii, think pineapple instead ;)