Welcome to WebmasterWorld Guest from 184.108.40.206
Forum Moderators: goodroi
You can disallow any resource, in fact what you are disallowing is any URL starting with that item so you could have
and it would disallow widgets.php, widgets.html, widgets.gif etc.
An interesting comment. What is the reason for this recommendation?
Dijkgraaf is spot on this one - to add some webservers seem NOT to issue redirect so bot won't get a chance to re-check new url (with slash) against robots.txt and thus unintentionally "violate" robots.txt. I had a few of these and ended up removing end slashes from robots.txt's disallow directivies to ensure that my bot won't crawl urls that webmaster clearly wanted not to be crawled even though technically it would have been webmaster's fault.
Not specifying slashes is the wisest way because it catches all possibilities.