Forum Moderators: goodroi

Message Too Old, No Replies

Remove part of your website

Should I use robots.txt or 301 redirects?

         

twist

7:27 pm on Dec 13, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I changed about 2,500 urls to make them shorter. I used a 301 redirect on them. MSN is fine, but both Yahoo and Google show both the new pages and the old pages. It's been about 4 months. Google has moved all the pages, old and new into supplemental index.

It seems that the 301 isn't working or is taking way to long. My website was doing great but now I only have about 20 pages not in the supplemental index. Most of my traffic is coming from the old url's.

[google.com...]

Should I use robots.txt to nix the old url's? Will this work?

I can't use the automatic URL removal system unless I allow the pages back and then 404 them. Which might cause problems with yahoo and msn. Anyone tried this?

Im also curious about removing pages like .css and error pages. Yahoo has both my css page and some error pages from url's that have never existed. Should I just ignore it?