Can I Remove Index.asp file from google

[flanok]

Hi

We are having problems redirecting index.asp back to root to prevent dublicate content issues because the site is on a shared server.

Unfortunatly G has indexed both the index.asp and root version of the site. I have no idea how the index.asp was found, as all links within the site go to root and I doubt anyone would link to the index page of the site.

But anyway it has happened, so I want to know if I can add the index.asp page to the robots.txt file and excluded it from googles index, or even better remove it from its search results, without it affecting the homepage of the site.

I am assuning that Google treats the root of the site as a different url to the index.asp of the site.

i.e Google treats www.example.com as a different url as
www.example.com/index.asp

So can I remove the index.asp version without it affecting the root version?

Many thanks.
Mark

[freejung]

This sounds like a perfect use for the canonical URL tag. [googlewebmastercentral.blogspot.com...]

I don't think it's a good idea to try to exclude index.asp using robots.txt, that might create confusion and risks removing the root URL as well.

[flanok]

Hi
Thanks for the reply.

I had thought about this, but the tag would also sow on the homepage (root page)?
Having the tag in a url that points to itself?
Would this work?

[rowtc2]

Do you don't have htaccess file? Ask hosting,all people have is these days htaccess for mod_rewrite.

If you have mod_rewrite enabled the job is simple.

Otherwise, use canonical tag. I do not use it because is a little confusing for me,but other people can help you.

I would not restrict in robots.txt to not exclude root index by mistake (theoreticaly this should not be happend, but who knows).

So, if you confirm mod_rewrite is used, job is simple.

[tedster]

Having the tag in a url that points to itself?
Would this work?

Absolutely yes - that IS the canonical form of the URL for that content. I have sites where we use the canonical tag on every page, even though there is no apparent duplicate problem with the URL configuration.

On a URL that is non-canonical, the canonical tag "works like" a 301 redirect, says Google. But on the preferred form of the URL, it just acts as a confirmation.

[TheMadScientist]

Ask hosting,all people have is these days htaccess for mod_rewrite.

Actually, .asp is Windows and must have ISAPI Rewrite installed on the server, which is much less common than mod_rewrite on Apache, otherwise, you're right and it's a fairly common, easy solution.

have sites where we use the canonical tag on every page, even though there is no apparent duplicate problem with the URL configuration.

Me too starting with my most recent project...
I couldn't think of any very good reasons to not include it, so I have been.

[g1smd]

First choice would be an ISAPI 301 redirect from /index.php to / and second choice would be the canonical tag to say that "/" is the correct URL.

[flanok]

Thanks everyone,

I think I will go with the canonical tag.

I had misunderstood where it could be used.
I had thought you put it on the pages you wanted redirected but omit it from the main page.
I now know different.

Thanks again

[freejung]

First choice would be an ISAPI 301 redirect from /index.php to / and second choice would be the canonical tag to say that "/" is the correct URL.

And there's no reason that I know of not to do both for good measure, just to make it absolutely clear that / is the preferred URL.

And yes, that is correct, it is perfectly acceptable for a page to have a canonical URL link that points to itself.

Here's what you don't want to do: don't set up a canonical URL link pointing to one URL, and then set up a 301 redirect pointing to some other URL. I did that by accident once and the page was completely removed from the index almost immediately -- fortunately it came back as soon as I fixed the problem.

[AnkitMaheshwari]

@freejung, the page should have disappeared due to 301 redirect after getting indexed.
IMO it has nothing to do with canonical attribute.