| How to stop robots on .php files
|
djtaverner
msg:1526886 | 1:16 pm on Feb 10, 2004 (gmt 0) |
I have a site and I want to stop google and all other robots finding the .php pages. All these pages have extended dynamic URLS. Is there any way I can shorthand the robots.txt to stop all these pages being indexed. Maybe: User-agent: googlebot Disallow: /*.php cheers
Dave
|
bakedjake
msg:1526887 | 6:20 pm on Feb 10, 2004 (gmt 0) |
| Is there any way I can shorthand the robots.txt to stop all these pages being indexed. |
|
Short of putting them in a directory called /php/ and excluding that directory, no.
|
Robert Thivierge
msg:1526888 | 6:40 pm on Feb 10, 2004 (gmt 0) |
GoogleBot will support doing what you want. Read their webmaster faq for details and an example (which is pretty similiar to what you did). However, other bots will not support this at all. There is no way to test such a non-standard robots.txt file. It is best to put your php files under a seperate directory (as bakedjake said). If you absolutely can't, you should at least use the ROBOTS NOINDEX meta tag, and also be sure PHP warning messages are turned off.
|
|
|
