Appspot proxy

Trying to block in htaccess

[crobb305]

There is a subdomain on the Appspot proxy that is indexed with my content. They display a #*$!ographic banner at the top of the page. I know the Appspot is on the Google IP range, so I don't know the best way to block. Any ideas?

The url indexed in Google has the form:

subdomain.appspot.com/http/www.example.com where example.com is my site.

I'd like to get it removed from the index to eliminate any problems with duplication. My domain has suffered enough in the past 6 months.

Any suggestions? I've been reading that appspot has caused a lot of headaches for some people. I think it's normally well behaved, but I don't like any version of my content to rank in Google, especially with inappropriate ads displayed at the top. The htaccess deny that I have always used to block other proxies is not working in this case.

[g1smd]

There are several dozen appspot proxies that Google indexes major chunks of the web through.

The proxy should have this in their robots.txt file.

User-agent: *
Disallow: /*/

You can block them with:

RewriteCond %{HTTP_USER_AGENT} (gcdnmirror|proxworx|morenames|andmore)
RewriteRule .* - [F]

Or you can get more creative and rewrite to a PHP script and send back whatever you want and they will display it. Things get interesting if you serve them mangled pages from a competitor. :)

[crobb305]

thanks! That was simple enough -- and I thought I tried that before, but it worked this time. For now I just returned a 403. I like your idea about doing the PHP script though :)

So far, I've only see one subdomain indexed, but if others appear, I will append as you have shown in the example.

Thanks for your help as always



C

[g1smd]

There's a great deal of fun you can have. :)