Welcome to WebmasterWorld Guest from 54.147.189.54

Forum Moderators: goodroi

Message Too Old, No Replies

How to stop robots on .php files

     
1:16 pm on Feb 10, 2004 (gmt 0)

Junior Member

10+ Year Member

joined:Nov 30, 2003
posts:125
votes: 0


I have a site and I want to stop google and all other robots finding the .php pages. All these pages have extended dynamic URLS.

Is there any way I can shorthand the robots.txt to stop all these pages being indexed.

Maybe:

User-agent: googlebot

Disallow: /*.php

cheers
Dave

6:20 pm on Feb 10, 2004 (gmt 0)

Administrator

WebmasterWorld Administrator bakedjake is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Apr 8, 2003
posts:3783
votes: 2


Is there any way I can shorthand the robots.txt to stop all these pages being indexed.

Short of putting them in a directory called /php/ and excluding that directory, no.

6:40 pm on Feb 10, 2004 (gmt 0)

Junior Member

10+ Year Member

joined:Oct 5, 2003
posts:59
votes: 0


GoogleBot will support doing what you want. Read their webmaster faq for details and an example (which is pretty similiar to what you did).

However, other bots will not support this at all. There is no way to test such a non-standard robots.txt file. It is best to put your php files under a seperate directory (as bakedjake said). If you absolutely can't, you should at least use the ROBOTS NOINDEX meta tag, and also be sure PHP warning messages are turned off.

 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members