homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Hardware and OS Related Technologies / Website Technology Issues
Forum Library, Charter, Moderators: phranque

Website Technology Issues Forum

How does Googlebot handle custom server response codes?

 11:55 am on Jun 24, 2009 (gmt 0)

I have run into an issue that I cannot find much relevant information on.

One of my clients is running 3rd party software on their site to prevent malicious hacker attacks. If the software identifies an attack (through HTTP), it will return a "301 Denied" response and then does a redirect. This instead of the tradition "301 Moved Permanently"

Does anyone know what Googlebot will "think" about this? (I hesitate to use the word respond or react as I assume the redirect will still happen.) We have had some issues with failed redirects in the past that have caused "Network Unreachable" errors. I need to make sure that Googlebot does not view this server response as invalid and flag the page as a "Network Unreachable" error.

If you are wondering how Googlebot will ever see this type of error, certain incorrectly marked-up HTML links to the site can invoke the defense code on the server.

I thank everyone reading this for their time and appreciate any thoughts/experience on the subject.




 12:25 pm on Jun 24, 2009 (gmt 0)

The "text" of the server response does not matter. It is the numerical code that has "meaning" to machines.

However, it is an error to return a 301 when a 403-Forbidden response is called for. Be aware that many (or most) "malicious" clients won't follow a redirect.



 12:49 pm on Jun 24, 2009 (gmt 0)

Thanks Jim. I had a feeling you would be the one to answer my question.

We had a 403 in there at first, but given our current situation we need a 301, as it is Googlebot that I am concerned about. Our security team are the ones who need to deal with the malicious clients ;).

Thanks again.


 1:17 pm on Jun 24, 2009 (gmt 0)

Non-malicious incorrectly-formatted links should be 301-redirected (if possible) to a corrected URL, and only maliciously-coded links (e.g. query string injection attempts) should get a 403. If it's not possible to correct malformed links, then a 404-Not Found is appropriate.


Global Options:
 top home search open messages active posts  

Home / Forums Index / Hardware and OS Related Technologies / Website Technology Issues
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved