Welcome to WebmasterWorld Guest from 54.224.113.125

Forum Moderators: goodroi

Message Too Old, No Replies

using robots.txt

is it necessary to exclude .php pages

     
2:13 pm on Jun 26, 2006 (gmt 0)

New User

10+ Year Member

joined:Apr 4, 2006
posts:31
votes: 0


Can anyone tell me if it is necessary or better to disallow .php files in the robots.txt file. I am concerned that the .php files in my site are affecting its ranking with the search engines? Will I get a better ranking if I disallow these files? Thanks
12:48 pm on June 27, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Jan 29, 2002
posts:980
votes: 0


No, it's not better or neccessary to disallow php pages in your robots.txt - especially if you want those php pages to appear in the search engines :)

Perhaps if you explain why you think your php pages are having a negative ranking effect someone here may be able to advise you further.

8:56 pm on July 1, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:July 3, 2002
posts:18903
votes: 0


The output of your PHP script is a page of valid HTML code. Spiders cannot see any difference in that HTML code than what they can see in the HTML code on a page that has a filename eding in .html or .anything else.
10:23 am on July 4, 2006 (gmt 0)

New User

10+ Year Member

joined:Apr 4, 2006
posts:31
votes: 0


Ok thanks very much both for that info. I am just trying to understand how spiders work as I am having a problem getting my .php sites high with Google and I am using a process of elimination to try and isolate the problem. I also thought I read that spiders dont like .php files and that some engines cant read them.

Am I better to include keywords in my .php files too then? Will they get read?

11:00 am on July 4, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Jan 29, 2002
posts:980
votes: 0


Do a view source from your browser on one of your php pages. That is what the spiders see. It's a html web page, exactly like any other web page, so can be optimised exactly like a static html page.

PHP runs on the server to generate the html code, and you have complete control of the html in exactly the same way as when you write a static html page in your editor.

The only potential problem I can think of is where php is set to append a sessionid onto the end of your urls. If you're not doing this, spiders/search engines will have no problems indexing your php pages.

11:53 am on July 13, 2006 (gmt 0)

New User

10+ Year Member

joined:July 13, 2006
posts:2
votes: 0


Can anyone tell me if its better to disallow my .php include files from search engines spiders? I have been told that Google does not like seeing two pages with exactly the same instances. Is this correct?