Googlebot and 301 Permanent Redirects

Strange 404 error caused by Google

[Bluestreak]

Hello all,

I created a subdomain titled photos.xxxxx.xxx, where I used an htaccess file to set a 301 PermDirect to my photos.php, which is located on my main domain.

Now for some reason when Google visits, it keeps looking for this file: "photos.phprobot.txt: just as it appears, resulting in a 404 error. I wonder how that happened? Did the 301 server screw up the Googlebot into looking for an erroenous photos.phprobots.txt file? I have no idea how this happened. I have no robots.txt or index.html in my subdomain root (which is pointless anyway because of the 301 redirect) Is this something Google will clear up on its own? It has ony been a couple of weeks since putting up the 301 redirect, and I understand it takes 6 to 8 weeks for Google to work through 301 redirects successfuly, so Im wondering if this is what it is.

Thanks for any help, and Happy New Year!

[msr986]

IMHO, there is something wrong with your redirect.

[jamesyap]

I think your .htaccess rewrite url syntax is wrong. I think it will mess up more files than the php file itself. Since gogglebot will 1st look for a file call robot.txt but your .htaccess has rewrite it to look for photo.phprobot.txt!

[Bluestreak]

My Redirect in htaccess looks like this:

RedirectPermanent / [xxx.xxxxx.xxx...]

I Xed out my domain, but other than that that's how it looks. I think the reason may be that I need to include an "index.html" line up there, even though it never existed, so it tells Google that the old file used to be index.html, and the new file is now photos.php, but Im just guessing.

[DaveAtIFG]

Poor old Googlebot... :) He drops by your subdomain (which he thinks is a domain) and dutifully requests robots.txt as any well behaved robot should, gotta see what to spider and what not to spider... Another day at the office...

Shazam! He suddenly finds himself whisked away to a different domain, [xxx.xxxxx.xxx...] "Whawazzat? Oh well, gotta get that robots.txt before I can start to work so I'll just ask for it at the new domain, [xxx.xxxxx.xxx...]

It's not clear to me why you need the redirect but it is clear that this redirect isn't quite adequate. Tell us what you're trying to accomplish and perhaps we can suggest a better soultion.

[Bluestreak]

I have a PHP driven site, and I created a subdomain to point to a specific page on that site that's not really related to the content on my primary domain. The problem is I cant pull that page out and place it in the subdomain's root directory, since it's integrated into the dynamically run site. All I can do is point visitors and bots using a 301 to that page. The subdomain is created mainly to give my clients a small and easy to remember web address. Without it, Id have to use the actual address, which wont fit on my business card :D

Right now Im testing it by including the "index.html" I mentioned before in the htaccess redirect, and include a modified robots.txt in my subdirectory where the page in question is located, might clear up the error.

[pardo]

We've got ip-address changed about three weeks ago. Googlebot is the only visitor on the old domain. I've read here that it could take 6-8 weeks before google refreshed it's own dns cache for a domain, but is there no other way to get the spider to the new domain.
In fact it is even more complicated for the pages on the old server differ in url-naming from the new servers pages for we redesigned our site and constructed other urls.

So what is the best solution to get 30 pages from www.domain.com/widgets.html on the old ip-address to www.domain.com/category-subcategory-widgets.htm into the new webserver?

Both servers are running windows iis. Old ip-address pages are page.html extensions and the new ones .htm

Hope someone can give some answers/help us out...?