Welcome to WebmasterWorld Guest from 184.73.8.127

Message Too Old, No Replies

Can I Remove Index.asp file from google

   
11:28 am on Apr 6, 2010 (gmt 0)

5+ Year Member



Hi

We are having problems redirecting index.asp back to root to prevent dublicate content issues because the site is on a shared server.

Unfortunatly G has indexed both the index.asp and root version of the site. I have no idea how the index.asp was found, as all links within the site go to root and I doubt anyone would link to the index page of the site.

But anyway it has happened, so I want to know if I can add the index.asp page to the robots.txt file and excluded it from googles index, or even better remove it from its search results, without it affecting the homepage of the site.

I am assuning that Google treats the root of the site as a different url to the index.asp of the site.

i.e Google treats www.example.com as a different url as
www.example.com/index.asp

So can I remove the index.asp version without it affecting the root version?

Many thanks.
Mark
5:22 pm on Apr 6, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



This sounds like a perfect use for the canonical URL tag. [googlewebmastercentral.blogspot.com...]

I don't think it's a good idea to try to exclude index.asp using robots.txt, that might create confusion and risks removing the root URL as well.
5:32 pm on Apr 6, 2010 (gmt 0)

5+ Year Member



Hi
Thanks for the reply.

I had thought about this, but the tag would also sow on the homepage (root page)?
Having the tag in a url that points to itself?
Would this work?
11:12 pm on Apr 6, 2010 (gmt 0)

5+ Year Member



Do you don't have htaccess file? Ask hosting,all people have is these days htaccess for mod_rewrite.

If you have mod_rewrite enabled the job is simple.

Otherwise, use canonical tag. I do not use it because is a little confusing for me,but other people can help you.

I would not restrict in robots.txt to not exclude root index by mistake (theoreticaly this should not be happend, but who knows).

So, if you confirm mod_rewrite is used, job is simple.
11:22 pm on Apr 6, 2010 (gmt 0)

WebmasterWorld Senior Member tedster is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Having the tag in a url that points to itself?
Would this work?

Absolutely yes - that IS the canonical form of the URL for that content. I have sites where we use the canonical tag on every page, even though there is no apparent duplicate problem with the URL configuration.

On a URL that is non-canonical, the canonical tag "works like" a 301 redirect, says Google. But on the preferred form of the URL, it just acts as a confirmation.
1:30 am on Apr 7, 2010 (gmt 0)

WebmasterWorld Senior Member themadscientist is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month



Ask hosting,all people have is these days htaccess for mod_rewrite.

Actually, .asp is Windows and must have ISAPI Rewrite installed on the server, which is much less common than mod_rewrite on Apache, otherwise, you're right and it's a fairly common, easy solution.

have sites where we use the canonical tag on every page, even though there is no apparent duplicate problem with the URL configuration.

Me too starting with my most recent project...
I couldn't think of any very good reasons to not include it, so I have been.
8:49 pm on Apr 7, 2010 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



First choice would be an ISAPI 301 redirect from /index.php to / and second choice would be the canonical tag to say that "/" is the correct URL.
10:08 pm on Apr 7, 2010 (gmt 0)

5+ Year Member



Thanks everyone,

I think I will go with the canonical tag.

I had misunderstood where it could be used.
I had thought you put it on the pages you wanted redirected but omit it from the main page.
I now know different.

Thanks again
4:38 pm on Apr 8, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



First choice would be an ISAPI 301 redirect from /index.php to / and second choice would be the canonical tag to say that "/" is the correct URL.


And there's no reason that I know of not to do both for good measure, just to make it absolutely clear that / is the preferred URL.

And yes, that is correct, it is perfectly acceptable for a page to have a canonical URL link that points to itself.

Here's what you don't want to do: don't set up a canonical URL link pointing to one URL, and then set up a 301 redirect pointing to some other URL. I did that by accident once and the page was completely removed from the index almost immediately -- fortunately it came back as soon as I fixed the problem.
5:37 am on Apr 9, 2010 (gmt 0)

5+ Year Member



@freejung, the page should have disappeared due to 301 redirect after getting indexed.
IMO it has nothing to do with canonical attribute.