Welcome to WebmasterWorld Guest from 188.8.131.52
I've searched G's Webmaster Galleria and 'Got Nothin' for helping on this issue.
Does anyone have an idea on how I can get a list of these site:mydomain.com URL's from Google?
[edited by: tedster at 1:25 am (utc) on Sep. 20, 2007]
[edit reason] switch to example.com - it can never be owned [/edit]
Changing the URLs of your pages is almost always a bad idea.
site:example.com/directory-a/I've done this and gotten the list of URL's but it has the accompanying verbage and then the URL's. I'm trying to get a strict list of URL's.
rewrite (not a redirect)I was thinking that 301 would be the way to go but you're saying that a mod-rewrite is better? I guess I will still need to make a list of old URL's and have the location of the new URL. A vast majority of these "new" pages are going to be right off the root and not in the sub-folder structure like they currently are.
You have to hunt around for a tool that will crawl our entire site, however. Some of them stop after a certain number of pages, and those tend to be the ones that rank highest in Google (go figure).
This is how I put together a link of 600 URLs for an .htaccess file so that I could 301 everything after an update to my site.
I have the list from Xenu and I have my site map list, but I am trying to figure out the differences in what Xenu and my sitemap say and what Google shows as indexed. Google has about 20% more URLS than Xenu shows or I send to them in my sitemap.
I suspect a problem.