homepage Welcome to WebmasterWorld Guest from 23.22.173.58
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Hardware and OS Related Technologies / Website Technology Issues
Forum Library, Charter, Moderators: phranque

Website Technology Issues Forum

    
Want to hide sub-domain from search engines
abdul



 
Msg#: 4489738 posted 10:09 am on Aug 30, 2012 (gmt 0)

Hi guys,

my apologize if this is not the right section to ask this question.


Abdul here from Pakistan. I am facing a problem with one of my websites. let me explain to you guys.

For example i have following site

www.abcdfgh.com and i use it just for one specific country (Pakistan). I set in code that if anyone came from different country it goes to following sub-domain

www.int.abcdefgh.com .. This part is working accurately.. the problem is

I don't want 2nd part (www.int.abcdefgh.com) of my website to be visible in any search engine for that specific country (Pakistan). currently when anyone search for my website from Pakistan, search engine shows some results from www.int.abcdefgh.com . . . . .

could anyone help me in this regard?


I will be gratefull. Thanks in advance

Abdul

 

phranque

WebmasterWorld Administrator phranque us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4489738 posted 1:01 pm on Aug 30, 2012 (gmt 0)

welcome to WebmasterWorld, abdul!

I set in code that if anyone came from different country it goes to following sub-domain

how do you identify a visitor from a different country?
how does that affect a visit from googlebot?

i'm guessing googlebot visits are identified as non-pakistan visits and are redirected to the www.int. subdomain and that gets indexed for the world. (all country indexes)


it is possible to geotarget the subdomain of a gTLD but it's not possible to "geo-exclude" content from a specific country's index.

you can forbid requests from a set of IP addresses.

i would suggest your best solution is to put your pakistani content on a .pk ccTLD and get that to outrank naturally your content on the gTLD.

OR...
create a pakistani subdomain of your gTLD and geotarget that content while keeping the rest of the world on the .com - essentially the reverse of what you are doing currently.

Leosghost

WebmasterWorld Senior Member leosghost us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 4489738 posted 3:00 pm on Aug 30, 2012 (gmt 0)

Or password protect the sub domain..and only allow international ( non Pakistani IP ) visitors to have a password..they can request it by email from you , directly or via a form on the login page on your sub domain..change the password every 7 or 15 days ..make each password specific to each request..

This is how most Pakistani companies deal with the problem of making and displaying "items" to international customers , that would cause "problems" for them, if "some people" at home in Pakistan knew that they were making such things..

This also allows them to check that it is actually a real, existent, legitimate trading foreign business making the request, and not a relative living abroad, of someone in Pakistan, who may be asking for access, just to see what they are "up to"..and maybe make problems for them, with what they find out..

HTH :)

Nearly forgot ..welcome to WebmasterWorld, abdul

abdul



 
Msg#: 4489738 posted 6:22 am on Aug 31, 2012 (gmt 0)

Thanks for your replies :-)

But real problem is not to hide it from Pakistani visitors... real problem is to hide it from google.com.pk

I mean to say my boss wanted

www.absdfg.com in google.com.pk

and www.intl.absdfgh.com for rest of the world...

currently we are using IP-Address which redirect foreign visitors from www.absedfgh.com to www.intl.absdfgh.com ... it is working but i don't know why google crawl that int. pages when i did not submit sitemaps, or did any kind of link building or provide search engine anything to crawl them.... except that redirect thing................

topr8

WebmasterWorld Senior Member topr8 us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 4489738 posted 6:33 am on Aug 31, 2012 (gmt 0)

googlebot is not visiting from a pakistani ip, therefore googlebot is being redirected to your subdomain.

so googlebot only sees your subdomain and never sees your main domain.

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4489738 posted 7:04 am on Aug 31, 2012 (gmt 0)

So far there only seems to be one googlebot, and it crawls from the US. You'd expect there to be one from every country, but nobody has seen them yet.

So if you want googlebot to see only the .pk version of the site, you have to put an exception to the redirect. If the visitor is from Pakistan OR from google, show them .pk. If the visitor is from elsewhere AND is not google, show them .int.

This may be a problem if google has already seen .int. And it might be a problem for people outside of Pakistan doing searches that lead to your site.

abdul



 
Msg#: 4489738 posted 7:14 am on Aug 31, 2012 (gmt 0)

lucy24


I agree with you. but for that i need to know search engines' bots IP addresses :-)

i know there are some service who find and sell those IP addresses. well guys thanks for your helpful participation. so far i end the redirection and set a different factor.. hope so it will work

phranque

WebmasterWorld Administrator phranque us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4489738 posted 10:46 am on Sep 3, 2012 (gmt 0)

regarding whatever your ultimate technical implementation is - if you serve googlebot differently from a human visitor it will likely appear as cloaking.

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4489738 posted 8:09 pm on Sep 3, 2012 (gmt 0)

Bottom line: You can do some geo-targeting. But there is no way to show different and mutually exclusive results in google.pk and google.somewhere-else, because it is all the same googlebot crawling from the same IP.

johnnymward



 
Msg#: 4489738 posted 6:20 am on Oct 1, 2012 (gmt 0)

set a robots.txt file in each site root folder which you to hide,tell search engines that you don't want your site crawled,put no follow attribute for particular domain of your site

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Hardware and OS Related Technologies / Website Technology Issues
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved