If you deliberately removed the pages, you should never have allowed 404s in the first place. You should be returning 410s. If it's a single complete directory, you're looking at one line in .htaccess or your config file. It's not too late to add it.
There is no known time limit, but 410s seem to go away faster than 404s.