homepage Welcome to WebmasterWorld Guest from 54.167.182.201
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / WebmasterWorld / Website Analytics - Tracking and Logging
Forum Library, Charter, Moderators: Receptional & mademetop

Website Analytics - Tracking and Logging Forum

    
302 Redirects and Google
Originally posted in the wrong place - Sorry I'm new!
Tempufugit

5+ Year Member



 
Msg#: 3107 posted 12:00 am on Feb 28, 2005 (gmt 0)

I currently use a virtual server to host my site. It runs Apache on a Sun Cobalt box and I don't have complete control over the behaviour of its server responses.

Can anyone explain the following server log entry?

66.249.64.66 - - [27/Feb/2005:16:12:11 -0500] "GET /robots.txt HTTP/1.0" 302 216 "-" "Googlebot/2.1 (+http://www.google.com/bot.html)"
66.249.64.66 - - [27/Feb/2005:16:12:12 -0500] "GET /robots.txt HTTP/1.0" 200 66 "-" "Googlebot/2.1 (+http://www.google.com/bot.html)"
66.249.64.66 - - [27/Feb/2005:16:12:12 -0500] "GET / HTTP/1.0" 302 206 "-" "Googlebot/2.1 (+http://www.google.com/bot.html)"
66.249.64.66 - - [27/Feb/2005:16:12:12 -0500] "GET / HTTP/1.0" 200 13443 "-" "Googlebot/2.1 (+http://www.google.com/bot.html)"

I am trying to prevent 302 redirects from occuring as I am under the impression that Google interpretes them as possible spamming!

Is the fact that I am running my site on a virtual server the reason that I get a 302 followed by the 200? I am NOT trying to fool googlebot by doing any sort of redirection.

This has only happened in the last couple of hours and I don't know if googlebot has returned to spider other pages. If it does it might happen when I am not watching - being the end of the month I won't be able to see the activity in the top 20 browser visits.

Confused
TF

 

davidof

10+ Year Member



 
Msg#: 3107 posted 11:24 pm on Feb 28, 2005 (gmt 0)

Google asks for robots.txt, the server sends a 302 response code telling google that the resource has move temporarily, Google then goes and fetches the resource. I assume from the logs that you are just redirecting to the same server which seems non-optimal to me. Presumably you have two domains and redirect one to the other. In that case a 301 redirect would be more logical. Have you been fiddling with your .htaccess file?

There is nothing wrong with 302, 301 and other redirects.

Tempufugit

5+ Year Member



 
Msg#: 3107 posted 12:42 am on Mar 1, 2005 (gmt 0)

Thanks David,

What is strange is that sometimes the robot gets a response of 200 straight away, other times it gets the temporarily moved code followed by the 200.

As I was saying I was wondering if the fact that my server is a "virtul" one? It was an inexpensive solution for hosting and I do not have full control over the server - for instance I can't edit the .htaccess file directly - even for security. To protect a directory I have a utility that writes to it although I can't see it!

I can't telnet to my server but ftp is no problem. My suspicion is that there is something strange going on. As I say I can't see the .htaccess file but I can see the other user's domains that are hosted on the same server! There is probably some redirecting to the same server as you suggest but there is nothing I can really do about that other than getting a proper hosting package or setting up my own server. (which I am thinking of doing).

Google did come back, but I have other suspicions that Google is being "put off" when it gets a 302 when it goes after my robots.txt file. If I had a META redirect on my base page that would look like a page highjack - especially if the wait time was set to zero. But there is no way that robots.txt can do this!

Anyway I was wondering if anyone else has seen this.

TF

Dreamquick

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 3107 posted 12:56 am on Mar 1, 2005 (gmt 0)

Depends on what it's requested...

For example if your site directs all requests for "example.com" to "www.example.com" (or vice versa) then you'd see two requests - one for the "wrong" URL which gets redirected, and then another for the "right" URL which gets delivered.

As for GB being "put off" by a 301/302 on your robots.txt file - I doubt you're the only person who redirects all traffic from one version of the site to another so there's no point worrying about it.

- Tony

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / WebmasterWorld / Website Analytics - Tracking and Logging
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved