Whoopsie - Dev domain has been crawled

[downhiller80]

I accidentally pasted a link to the dev copy of my site in a forum, instead of the live copy.

The result? Google has now indexed 13,000 pages of it!

I've added a robots.txt to block further access, and also taken the site down so any and every request gets a 404.

Is that the best I can do? Is there a way I can tell google to drop the whole domain ASAP? Should I 301 redirect to the homepage of my live site instead of 404ing?

I can't 301 redirect to specific pages on my live site as the url structure is different, as I hadn't uploaded the new version to this particular dev URL for a while, so all of the links are different to the new site. I COULD make it work, but it'd be a lot of hassle.

What would you do?

cheers

[tedster]

Is there a way I can tell google to drop the whole domain ASAP?

Yes, but first you must set up a Webmaster Tools account for the domain and verify your ownership. Then, under Site Configuration > Crawler Access you can use the option that says:

Your entire site
Remove your site from appearing in Google search results.

[downhiller80]

Thankyou! I did look in WT and somehow missed that, I thought the URL removal tool there was only for singal URLs. Perfect, thanks. Done.