homepage Welcome to WebmasterWorld Guest from 54.225.57.156
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Google / Google SEO News and Discussion
Forum Library, Charter, Moderators: Robert Charlton & aakk9999 & brotherhood of lan & goodroi

Google SEO News and Discussion Forum

    
How to stop my site showing up on non-US Google results
DLadybug




msg:3625161
 5:40 am on Apr 12, 2008 (gmt 0)

Sorry.. I searched all over the SEO forums and google sub forums, and got lost someplace in the robots.txt discussions.

I need help refining what area to post in because I don't know if I have a htaccess issue, robots.txt issue, or what.

First, I want to make clear, I am not worried about bringing my page rank up at all, I just want to stop showing up on non-US google results. So this isn't about SEO.

I have a site serving the SE United States. However, 80% of my visitors come from Ireland, Israel, China, Japan, Italy, South America, and of all places, Botswana?

I used some google tool (I can't remember which) that showed my page rank on non US google indexes varies from 4 to as high as 7.

That would explain the off-continent percentage, because I have a white bar / page rank 0 here. So...

1. Is there really a different feed for different countries?

2.Can I keep google indexing for US users but not others?

3.Is it one googlebot gathering data for all places, or are there different ones? (That would explain why there is some google-thing on my msg boards every day. Or there is one google bot and he lives at my place.)

4.To block them or it, do I have to know the name of every bot and spider by IP or nickname or whatever?

5.Is that done by htaccess or robots.txt?

Right now, I have this massive htaccess file blocking non-US IPs, but it's causing huge lag on my msg board, so I'm hoping if I can just stop showing up in results that attract these users, I can stop the problem before they show up at the door.

Bless anyone with the patience to help. :)

 

Marcia




msg:3625166
 6:07 am on Apr 12, 2008 (gmt 0)

1. Is there really a different feed for different countries?

2.Can I keep google indexing for US users but not others?

3.Is it one googlebot gathering data for all places, or are there different ones? (That would explain why there is some google-thing on my msg boards every day. Or there is one google bot and he lives at my place.)

4.To block them or it, do I have to know the name of every bot and spider by IP or nickname or whatever?


There are technology issues involved, but it's basically more of a Google indexing issue than anything else.

[edited by: Marcia at 6:10 am (utc) on April 12, 2008]

DLadybug




msg:3625216
 8:16 am on Apr 12, 2008 (gmt 0)

Thanks for moving me, my owner wasn't sure where I belonged
:)
=================

Ok, I found something, there is a function in google tools to "set geographic target" and US audience is an option. I'll start here.

But this doesn't say it will restrict results to only that geo-region, so most of my original questions still stand. I'm not looking for spoon-feeding, just filling in the huge gaps I have about spiders and all. I have't been following it closely since I dropped all active SEO efforts in about 2003.

I decided to just go natural and SERP would develop on the merits of content. Who'd have thunk Anyone would be looking for a way OUT of search results?
==========

Marcia, thanks for the swift reply.. that pointed me to the google webmaster tools stuff, a great start!

[edited by: DLadybug at 8:18 am (utc) on April 12, 2008]

lammert




msg:3625249
 10:16 am on Apr 12, 2008 (gmt 0)

Geographic targetting boosts ranking in one region compared to others, but it doesn't remove a site from foreign search results. My experience is that it has no greater power than a country TLD like .de for Germany or .fr for France, or hosting your site on an IP adddress which is locate in the country to target.

1. Is there really a different feed for different countries?

No, every Google datacenter can produce the results for all countries and languages in the world by just changing a few parameters in the search URL. Google tries to sort the results based on relevancy, matching languages and geographic origin of incomming links to a site, but in principle every URL can appear in every SERP on every visitors location. There is no such thing as totally separate feeds.

2.Can I keep google indexing for US users but not others?

3.Is it one googlebot gathering data for all places, or are there different ones? (That would explain why there is some google-thing on my msg boards every day. Or there is one google bot and he lives at my place.)

There is just one Googlebot crawling for all countries and data centers. If you are on one Google data center, you are practically speaking in all, because they exchange pages on the fly. There is even datasharing behind the scenes between differen Google spider technologies. If Googlebot doesn't visit a specific URL but Mediabot which is used for AdSense ad matching is, the pages fetched by Mediabot may be examined and used by Googlebot.

There is no way you can block your site from showing up in Google results for one country and not for others, unless your site is China related and happens to trigger a filter in Google's China firewall.

The only way to tackle this reliably is to block the foreign visitors at your door, i.e. use some form of geotargetting where you map the IP address of the visitor to a geograhical location and allow or deny access based on that. But geotargetting is not 100% reliable, especially with some larger ISPs like AOL which use a handful of proxies for all their customers and you may end up with some foreign visitors slipping through, and worse, a number of legitimate visitors who can't connect anymore.

My advice is not to fight the battle against foreign visitors, but to monetize the traffic. Many people are figting for traffic and you--wanting to kill 80% of your traffic because it doesn't match the current content of the site--are really an exception. Why not monetize this traffic in some way instead of blocking people? This is free traffic which is in principe targetted audience, based on that they found you through Google search, and not some form of shady traffic generation scheme.

<off topic>
Botswana is a very nice country to visit. No reason to block them :)
</off topic>

DLadybug




msg:3625267
 11:49 am on Apr 12, 2008 (gmt 0)

Thanks for so much helpful info!

I know it sounds incredible to want to limit access so, but a social-type site in one southern US city does not need members who can't attend our functions. This is like a widget-repair service in Iowa being swamped by queries from Norway.. nice to get the calls, but no way to actually DO any repairs, and local customers can't get a call through.

RE visiting Botswana, It's an idea! I didn't realise how my Q could be taken, but rest assured, it's not origin of the user that is my concern, only proximity to the events we're advertising. We do have international members that regularly visit the area, who found us via word of mouth rather than google search.

Honestly, I'm also concerned there is a security expoit or something attracting them.. I have no human graphics on the site, no displayed member e-mails to harvest, so what do they want that they come back to register day after day?

Ok. So. Steps taken.. I've done the geo-targeting via google tools. I've changed membership registration to manual admin-validating. I'll try flipping that htaccess file off occasionally and check to see which of them returns.. I know there are several persistant IPs, I'll try to ban those primarily and maybe if I can stop the top 20 I can drop the "ban all except" htaccess file.

Final ettiquette question, is it considered rude / topic-bumping to report back on this thread after trying some of these things? Or is it adding helpful info for the next person?

tedster




msg:3625275
 12:05 pm on Apr 12, 2008 (gmt 0)

You're welcome to report back to the thread in the future.

what do they want that they come back to register day after day?

That could very well be an automated bot, looking for a way to send spam through your form, or trying to find a security hole to hack into your server. There's a lot of that going around, unfortunately, trying to place various things on web pages - hiding links, or various kinds of badware downloads in what some people call "parasite hosting". The best defense is keeping your technology patched.

By the way, about your principal question - consider that some of your local members may travel internationally at some point and want to access your site while outside the US. I don't think you'd want to block them in any way with your solution to non-US traffic.

rytis




msg:3625282
 12:45 pm on Apr 12, 2008 (gmt 0)

I have some evidence that Google started to mix the location of incoming links into algo. If this works well for Google, your situation should improve over time, presumably you get links from area (or at least country) which you target.

Marcia




msg:3625332
 3:04 pm on Apr 12, 2008 (gmt 0)

>>mix the location of incoming links into algo

It's been hinted at before, and it's very likely.

For anyone who might have the endurance to wade through it (this one is wordier and more obtuse than most), there's a patent application that's been published that might have some bearing on geographic factors:

ENTITY DISPLAY PRIORITY IN A DISTRIBUTED GEOGRAPHIC INFORMATION SYSTEM [appft1.uspto.gov]

Afterthought:

I have a site serving the SE United States.

If you have a business location located in that area, if you haven't submitted to Google Maps, it might be a good idea to do so to help establish physical locale. And/or if there are events, adding Google Maps for instructions to the location of the events could also help with identifying geographic targeting.

From the Patent Application:

A geographic information system (GIS) is a system for archiving, retrieving, and manipulating data that has been stored and indexed according to the geographic coordinates of its elements.

[edited by: Marcia at 3:23 pm (utc) on April 12, 2008]

Robert Charlton




msg:3625941
 6:40 pm on Apr 13, 2008 (gmt 0)

However, 80% of my visitors come from Ireland, Israel, China, Japan, Italy, South America, and of all places, Botswana?

Do you know these are visitors coming in via Google search? I think you need to analyze your logs and determine whether these are in fact human visitors, or whether they're bots. It sounds to me like they might be bots.

Conceivably, your message board is built on a platform that might have a vulnerability which rogue bots like to target, and they've found you via a software or template signature of some sort.

For the human visitors, including your locality in your title and on the page might help attract the people you want to visit and discourage some click-throughs from the others.

incrediBILL




msg:3625988
 9:03 pm on Apr 13, 2008 (gmt 0)

Without seeing your log files it's hard to determine what's really going on but it's highly likely a lot of strange traffic is bots trying to scrape and or spam your site, including hack attempts.

If you strictly serve the United States the best solution for you would be to ALLOW only IPs assigned to ARIN.NET (American Registry for Internet Numbers) in your .htaccess file and DENY all others.

wilderness




msg:3626001
 9:57 pm on Apr 13, 2008 (gmt 0)

DladyBug,
A very simple beginning for you localized goal (SE US) would be to deny these ranges:

RewriteCond %{REMOTE_ADDR} ^(189¦190¦19[3-6])\. [OR]
RewriteCond %{REMOTE_ADDR} ^(20¦20[0-3]¦209¦21[0-3])\. [OR]
RewriteCond %{REMOTE_ADDR} ^(5[789]¦6[012])

(Please note; the forum breaks the pipe chracter and requires correcting before use.)

These will need monitoring and there are many more, however I'm simply not goiny to provide such a list in an open forum.

In addition you may deny referrals based on the two-letter prefix that leads or follows domain names.

Non of these denials affect you major SE listings with the exception being a major SE that would crawl from one of these denied ranges (which your not focused on non-US listings anyway.

Don

incrediBILL




msg:3626007
 10:05 pm on Apr 13, 2008 (gmt 0)

In addition you may deny referrals based on the two-letter prefix that leads or follows domain names.

Such as blocking referrals from "google.ie", "google.ca", etc.

wilderness




msg:3626009
 10:12 pm on Apr 13, 2008 (gmt 0)

Such as blocking referrals from "google.ie", "google.ca", etc.

Yes, however you need not restrict it to a specific SERP or domain name and you may add exceptions to allow-in the countries you desire.

As an aside; "ca.google" is also a variation for any domain, although not specifically with google.
The one problem I've encountered with this implementation was in online email tools, as related to group discussions such as LadyBugs foum. Many of these online email tools utilize leading or trailing two-letter suffixs'.

wilderness




msg:3626015
 10:21 pm on Apr 13, 2008 (gmt 0)

In reflection, I should add some caution!

I have no idea how these actions affect click-thrus or Google Ads, or PR, or site maps, any other item similar item.

They certaianly do not affect the standardized sipdering of your pages and/or the resulting entry of the crawl results.

Don

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google SEO News and Discussion
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved