Disallowing by date.

Is it possible to keep some page and disallow new ones...?

[bouncybunny]

I want to disallow the indexing of URLs which have a certain structure in the URL, but I only want to apply the disallow rules to newly discovered pages. I.E. I don't want URLs that are already indexed to be dropped.

I'm guessing that I am on to a loser here, but is this possible using robots.txt?

[goodroi]

disallow the indexing of URLs which have a certain structure in the URL

this is why people use wildcards in the robots.txt

only want to apply the disallow rules to newly discovered pages

robots.txt can not be applied based on time. htaccess might have the flexibility you are looking for

[bouncybunny]

Thanks.

I thought so.

.htaccess is probably getting beyond my level of technical ability.