Forum Moderators: open
I've been trying to find the answer to this for a long time now. My site got into google about two months ago, and it's great, I'm getting lots of relevant hits. The only problem is that it has never crawled past my index.html page.
Every time googlebot comes (for a while it was weekly, now it seems more like monthly), it grabs my robots.txt page, and my index.html page quickly in succession, but then never grabs anything else.
Does anyone have any suggestions as to what I may be doing wrong.
Is my robots.txt file correct?
Is it because I am using PHP session tracking, and it tends to append a "?PHPSESSID=f4578370d754c1005ad08229eeffd24c" to every URL. If this is the case, does anyone know how to get PHP not to do that?
Or are there any other reasons anyone can think of as to why this may happen?
Thanks in advance....
Cam
[edited by: Marcia at 5:31 am (utc) on July 4, 2002]
[edit reason] url removed [/edit]
Your robots.txt looks ok. It is your dynamic links. I believe you can use mod_rewrite to fix your problem. do a site search at the top for mod_rewrite. I am not a unix guy. Maybe someone else can give you a better fix for your ? urls.
Just kidding! ;)
You can check your robots.txt with the robots.txt validator [searchengineworld.com].
>tends to append a "?PHPSESSID=f4578370d754c1005ad08229eeffd24c" to every URL. If this is the case, does anyone know how to get PHP not to do that?
That's a scripting issue I can't answer, but it's a problem. There needs to be some regular, normal navigation to be followed.
Is there any particular reason why that type of linking is necessary?
Basically, if it's Google that is requesting the page, I'm turing off the PHP sessions, and hence the crazy "url?PHPSESSID=asdfqwer" stuff should stop as far as Google is concerned.
I'll wait and see, as that's really all I can do, isn't it!
