Welcome to WebmasterWorld Guest from 184.108.40.206
Forum Moderators: goodroi
My robots.txt reads...
Will this also block www.example.co.uk/test.aspx?
What exactly does this do?
I think the old website was done in ASP & it is still live so the guy before me wrote this tobots.txt file.
Any help would be great!
If the pages are no longer required, the safe and sensible thing to do is delete them - the web has quite enough clutter; also, particularly if any links exist to the unwanted pages, they may still appear in the serps as URLs or supplementary results.