Welcome to WebmasterWorld Guest from 220.127.116.11
Forum Moderators: goodroi
I think I am shooting my self in the foot with the robot.txt file I have on my website
Here is my question :
I use to have a webpage that I created in my back office named www.mywebpage.com/p123/mywebpage.html
I restricted this page with my robot that.txt file because it doesn't exist anymore
and my new new page address is www.mywebpage.com/p234/mynewwebpage.html ( the way my website is built the .html name doesn't matter only the number matters ) !
but I noticed that when I type the old address that I thought was restricted it brings me to my current page and my webpage has 2 different adress for the same page with 2 different numbers ?
I don't understand why ? what should I do should still restrict the old page or should I not restrict anything and let google figure out the right page ?
i think that you should restrict your old page, so that google may not figure it out.if google figured it out then it will create some problems, like google may ban your site or apply some penalty.
So far i have visited a site that has two url's of same page, but google still showing on the top position in SERP's.
my old page is [myoldpage.com...]
and my new page is
The only thing that changes is the .html but in my case my what only counts on my website is the number P123
My guess is I shouldn't restrict anything and just let google do his thing and change the .html title when it finds the new one don't you think ?
The redirect should work whether the old URL was requested as www or as non-www, and the redirect should force the new URL to have the www in it... otherwise that is another form of Duplicate Content.
However, as pointed out above, the proper approach is to 301-redirect the old URL to the new URL, and robots.txt should not be involved.
After this primary issue is resolved, do take a look around here (using the WebmasterWorld site search) for threads on "duplicate content." Your statement that "in my case my what only counts on my website is the number P123" indicates that the same content will be returned for http://www.example.com/p123/<anything>, which is a recipe for a duplicate-content disaster, especially if one of your competitors discovers this vulnerability and exploits it to create a bunch of non-canonical links to your pages in order to destroy the ranking of your canonical URLs...