homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

Google gone crazy?

 9:43 am on Aug 22, 2008 (gmt 0)

Normal Google IP, UA and after reading robots.txt a direct request for www.example.com/fdfdkll.html

Any ideas?




 6:50 pm on Aug 22, 2008 (gmt 0)

Try a google on the "page" URL alone (minus domain).
You'll be provided with a solitary result, with insight.


 11:08 pm on Aug 22, 2008 (gmt 0)

I found this same file name in my 404 log yesterday, and did a search on that name. I got no results at all. After reading this, I did the search again and got several results, but I'm not sure that I got the insight.

Is this just Google doing a random check to see what happens? That's the closest thing I could find to an explanation.


 12:38 am on Aug 23, 2008 (gmt 0)

Google does often test sites with page names designed to deliver a 404 response to see how your server handle the 404 response.

Some sites return a 200 OK with some other page, typically the home page, to stop visitor bounce and Google identifies this so it doesn't index the same page multiple times and possible make the wrong page supplemental.


 12:39 pm on Aug 25, 2008 (gmt 0)

Thanks for this post - I just found this same file in my logs and would have spent hours otherwise trying to figure it out

Global Options:
 top home search open messages active posts  

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved