Welcome to WebmasterWorld Guest from

Forum Moderators: goodroi

Message Too Old, No Replies

Exculde php url with vars using robots.txt

Exclude urls with vars using robots.txt



5:13 pm on Nov 3, 2005 (gmt 0)

10+ Year Member


I have a script that tracks downloads. The URL is in this format :


What is the correct command to restrict bots from spidering that kind of file using robots.txt?
Will "Disallow: /dl.php" be enough?

Thank you


7:44 pm on Nov 3, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

Yes, that should be enough.

Featured Threads

Hot Threads This Week

Hot Threads This Month