Software/script to gather all URLs listed in a search engine

does it exist, what's it generally called

[hannamyluv]

I have a website that I am going to restructure. Is there a software or script I can run that will gather the indexed URLs so that I know which ones need to be 301'd

I don't need specific names, just a good nudge as to what this would be called so that I can go search for it.

[Blackie]

If we take Google:
site:www.yourdomain.com

Yahoo:
domain:www.yourdomain.com

[hannamyluv]

Well, yes, I can do that, but copying & pasting all 100+ URLs would be a bit of a pain. How about something that puts all the URLs in a nice easy to use file?

[Blackie]

I see. I never heard of software like this.

It is easy to set up a small php (perl, or whatever you like) script to parse through the results and make a file with all the links for you. Tip 3-5 hours work.

Remeber to use API to respective SE. The only sad thing about this approach is that I think one is getting slightly different results from API's then from search engine itself.

I hope others can enlighten us on that one :-)