301 redirects and robots.txt

[lpiracing]

Our site has changed it's URL structure from www.example.com/pc to www.example.com/store/pc.

I know it's not ideal to add another category, but in order to be able to easily upgrade our cart system we need to keep their default URL structure. We did major mods to remove the store directory in the old system, and never really did get everything working right.

So I 301 redirected the www.example.com/pc pages to the new www.example.com/racing-parts/pc structure. It appears Google ranks all the old urls higher than the new ones, I suspect due to the extra directory away from the root.

So when should I add a dis-allow the www.example.com/pc to the robots.txt to get rid of the old URL's.

Any other thoughts on how to get Google to rank the new pages over the old ones?

We have a sitemap, etc...

[tedster]

You should never need to add it. As Google finds the 301 redirects those URLs will naturally drop from the index.

Google has been a bit chaotic in recent weeks, doing some back-end changes to prepare for their new infrastructure. So lots of general spidering and housekeeping is behind schedule. As long as the visitor who clicks on the old URL is getting 301 redirected, you have no worries.

[seolearner]

Yes, I have also tried 301 redirect. I needed to change the file name. So I set 301 to new file name. Until Google crawled my old URL again, I was getting copied content warning message in WMT. But as soon as my old URL was crawled again, the old URL was dropped from the index and also the warning message in WMT disappeared. So wait until your old URLs crawled again.

[Robert Charlton]

So when should I add a dis-allow the www.example.com/pc to the robots.txt to get rid of the old URL's.

If you had added the disallow for the old urls, in fact, Google most likely would have been slower about dropping them, as Googlebot wouldn't have been able to crawl them and discover the 301s.