Checking for 404s Before Site Change

is there an easy way?

[iJeep]

We are preparing to change shopping carts for our site. We have created a database and script to redirect old URLs to new ones, where needed. Before we transition the new cart to the live server, I would like to check to make sure there are no links that have been missed.

I thought about using Xenu to crawl the old site then changing the hosts file to redirect the requests to the new one....but I don't see how to recheck links in Xenu.

Is there a way to do that or an easier way to check for future 404 errors?

[ergophobe]

you can use the W3c link validator. I just used that as part of the same process your doing - it's a long process, but it gets you there

[iJeep]

That looks like a service similar to Xenu. I'm looking for something that will take the addresses of all public urls at my current site and check them against a different (test) domain to see if there will be any missing pages after the transition.

[ergophobe]

Ah, got it. How about this - get Xenu to generate lists of URLs on both site, sort alphabetically, then run through diff and see what's different.

[aakk9999]

I presume you have a test (staging) site with a new cart somewhere?

I use a different tool which can export all internal links that it finds on the page as HTML Sitemap.

What I do is:

1) Run this tool against the current (Live) site and export the HTML sitemap

2) Edit HTML (by doing global find/replace) to change Live domain into Test domain

3) Temporarily link this changed HTML sitemap onto TEST domain, directly from the home page

4) Run the same tool again against the TEST domain. It will find added HTML sitemap and will try all its links. They should be all returning 301 Redirect

The tool is very similar to Xenu, only its output is a bit less "messy".

[edited by: ergophobe at 12:48 am (utc) on Jan 26, 2011]