Welcome to WebmasterWorld Guest from

Forum Moderators: goodroi

Message Too Old, No Replies

using robots.txt

is it necessary to exclude .php pages



2:13 pm on Jun 26, 2006 (gmt 0)

5+ Year Member

Can anyone tell me if it is necessary or better to disallow .php files in the robots.txt file. I am concerned that the .php files in my site are affecting its ranking with the search engines? Will I get a better ranking if I disallow these files? Thanks


12:48 pm on Jun 27, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

No, it's not better or neccessary to disallow php pages in your robots.txt - especially if you want those php pages to appear in the search engines :)

Perhaps if you explain why you think your php pages are having a negative ranking effect someone here may be able to advise you further.


8:56 pm on Jul 1, 2006 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

The output of your PHP script is a page of valid HTML code. Spiders cannot see any difference in that HTML code than what they can see in the HTML code on a page that has a filename eding in .html or .anything else.


10:23 am on Jul 4, 2006 (gmt 0)

5+ Year Member

Ok thanks very much both for that info. I am just trying to understand how spiders work as I am having a problem getting my .php sites high with Google and I am using a process of elimination to try and isolate the problem. I also thought I read that spiders dont like .php files and that some engines cant read them.

Am I better to include keywords in my .php files too then? Will they get read?


11:00 am on Jul 4, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

Do a view source from your browser on one of your php pages. That is what the spiders see. It's a html web page, exactly like any other web page, so can be optimised exactly like a static html page.

PHP runs on the server to generate the html code, and you have complete control of the html in exactly the same way as when you write a static html page in your editor.

The only potential problem I can think of is where php is set to append a sessionid onto the end of your urls. If you're not doing this, spiders/search engines will have no problems indexing your php pages.


11:53 am on Jul 13, 2006 (gmt 0)

5+ Year Member

Can anyone tell me if its better to disallow my .php include files from search engines spiders? I have been told that Google does not like seeing two pages with exactly the same instances. Is this correct?

Featured Threads

Hot Threads This Week

Hot Threads This Month