Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Google replaces ? and = with %3F and %3D then reports crawl error

         

aakk9999

12:33 pm on May 16, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I have two URLs in the GWT crawl error section where the ? and = in URL have been replaced with %3F and %3D. When such URL is requested, it returns HTTP 400.

E.g.

www.example.com/module.aspx?param=something
is shown in GWT as
www.example.com/module.aspx%3Fparam%3Dsomething

When I hoover over this URL in GWT, it shows the correct URL, however, when I click on it, it returns 400.

The top URL has now been rewritten to static. I tried to redirect the second one (with % in it), but it seems I am unable to do so and HTTP 400 is still returned (IIS environment, custom rewrite module).

Has anyone else had similar errors and if I am unable to do the redirect then what is the second best to sanitise this? Should I just not worry or stop this string in robots.txt?

Interestingly, from thousand dynamic URLs, only very few do occasionally reported like this in GWT.

enigma1

1:08 pm on May 17, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



You need to find which script is causing this encoding problem. The first link is the correct one the second should give a 404 or some other error handler subject to the rules you have on your server.

The fact it is reported it may imply the bot found it in another page of your site. Check the server logs see how the link and google access shows up. It may give you some hints on previous entries what was accessed previously and you may be able to replicate it.

I also remember cases where the GWT was reporting incorrectly URLs (non-existing ones) because it was trying to access them via a proxy. If that is the case you could ignore it.