redirect

[scorpion]

What apache server code should you use for a redirect such that google will not "see" it and read the resultant page as the URL requested? I mean, I want to do a transparent redirect in such a way that google cannot see a difference between the URL specified and the new location (I am using mod_rewrite)...

[jdMorgan]

scorpion,

To redirect transparently from visible.html to invisible.html, just do

RewriteRule ^visible\.html$ /invisible.html [L]

Jim

[bcc1234]

RedirectPermanent

that's 301

[scorpion]

thanks! In your example, would google spider

mydomain.com/visible.html as the domain in the search results, but return invisible.html when the user views it? I mean, google won't list in the search result URL mydomain.com/invisible.html?

[msr986]

With a ModRewite, the server "rewrites" the URL "behind the scenes". The user agent never knows the content was rewritten, and thinks it's getting the original request.

Therefore, in the example above, Google and any other requesting user agent will only see visible.html.

[jdMorgan]

scorpion,

As far as anyone but you is concerned invisible.html does not exist. However, whenever visible.html is requested, the contents served to the requestor actually come from the file /yourserverpath/invisible.html, and it is visible.html that actually doesn't exist as a file. visible.html is effectively an alias for invisible.html.

This is quite useful for providing short, search-engine-friendly URLs for nasty, long URLs created by scripts and such.

Jim

[scorpion]

that makes sense. How would google spider a top-level domain abc.com if

index.php calls index.html and index.html is rewritten to welcome.php?

Will google just list it as 'http://www.abc.com' or will it see it as 'http://www.abc.com/index.html' (if it spiders the last case, it will bypass index.php all-together!')

[jdMorgan]

scorpion,

It all depends on how you define "calls" and "rewrites". You can set this up any way you like, as long as you are careful. If you aren't careful, then you'll get a few "screwy links" floating around out there on the 'net, but it's not the end of the world.

Jim

[bcc1234]

I think the problem with rewrite module is if google somehow gets to request invisible.html directly - you'll end up serving duplicate content.

I've read some posts about google requesting URLs that are not linked from anywhere else.

[jdMorgan]

Search engines requesting invisible.html directly is not really a very big problem...

# Rewrite requests for visible.html to invisible.html, [L] flag forces mod_rewrite exit
RewriteRule ^visible\.html$ /invisible.html [L]
# Respond to any direct http request for invisible.html with 403-Forbidden 
RewriteRule ^invisible\.html$ - [F] 

-or-

RewriteRule ^invisible\.html$ - [G] 

-or-

RewriteRule ^invisible\.html$ /visible.html [R=301,L]

As long as the rules are in the order shown, the first rule has an [L] flag, and your site never makes any http (URL-based) requests for invisible.html, this will block or redirect the unexpected requests. That is, your on-site scripts can access invisible.html by file pathname, but not by http URL. You can add qualifiers to the secondary rule to allow restricted http accesses and add blocks for SE UAs and IPs if you really need to lock it down.

Publicly-accessible server log files, unintentionally-published links, and visits to your own sites' "secret" pages with the Google Toolbar Advanced Features enabled are the most likely explanations. If there is no link, then excepting a Google Toolbar PR check, Google can't find it. If they do find it because of one of the above accidents, just rename the file to nonvisible.hmtl, change the rewrite rules, and you're good to go again.

However, if you have visible_1.html through visible_99.html all pointed to invisible.html, then yes, that is duplicate content, and most if not all of them will get a penalty and/or disappear.

Jim

[bcc1234]

If nobody ever requests invisible.html directly then what's the point?
Why not just rename you invisible.html to visible.html and be done with it?