Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Bot adds / to query string

Google and Yahoo

         

Staffa

8:25 pm on Aug 14, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I have 300+ dynamic pages that both bots are happy to index.
In the last couple of days I noticed Googlebot requesting a handful of already indexed pages and adding / after the query string, i.e. page.asp?prodID=233/ and creating a server error.

Today, Yahoo did the same with a couple of pages.

Does anyone has any idea why both bots do that?

Many thanks in advance.

Staffa

6:56 am on Aug 16, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Surely, someone here could shed some light on my question?

webspy

9:48 am on Aug 16, 2005 (gmt 0)

10+ Year Member



I have all my URLs with a / at the end and Yahoo just stripped this / out on every URL. I dun understand why.
When you look through the Yahoo SERP now, every URL you click is without / at the end and will redirect via 301 to the right version with /. Any idea why Yahoo does that?

softwareengineer99

10:45 am on Aug 16, 2005 (gmt 0)

10+ Year Member



adding / after the query string, i.e. page.asp?prodID=233/ and creating a server error.

What about rewriting URLs using mod_rewrite etc?

Staffa

1:03 pm on Aug 16, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Thanks for the suggestion softwareengineer99
I am on win server and the dynamic pages are indexed fine

I just wonder why G and Y add a / to a few - well so far - of the query strings when they revisit. There is no obvious reason for doing so.

dmorison

1:07 pm on Aug 16, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



If they're both doing it my guess is that somewhere somebody or something has created an invalid link to your site with the trailing forward slash...

Staffa

3:40 pm on Aug 16, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



dmorison you may very well be right about this.

I looked over my logs and in fact the request of each of the bots is for two different pages with querystring/ per bot. They just requested these pages several times.

I tried a search for the URL but / is ignored even when between " " .
I guess I'll just have to wait till wherever the bots get the link from is sending a visitor directly or something else comes up that clears it all up.

Many thanks everyone for your input.