homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Google / Google SEO News and Discussion
Forum Library, Charter, Moderators: Robert Charlton & aakk9999 & brotherhood of lan & goodroi

Google SEO News and Discussion Forum

Googlebot is crashing my site
googlebot is on an infinite loop of one of my pages, help!

 4:46 pm on Jan 14, 2007 (gmt 0)

I run a site that has over 2000 pages. Normally we are crawled thoroughly and wonderfully everyday. We have five sections of archives pages. Last night googlebot got stuck on the first page of each of our five entry archive pages. Googlebot is stuck on an infinite loop of crawling those pages and we can't stop it. It is not going anywhere else (other than the homepage, which it goes to every 5 minutes) on the site, just those 5 pages and the homepage. We put a meta name='robots' content='noindex,nofollow' tag, it didn't stop it, we put a meta name='googlebot' content='noindex, nofollow', it is still stuck and we don't know what to do. It is not following any of the links on any of our pages and the constant loop is causing us to lose memory and we keep getting shut down. Any idea what the heck is going on?



 9:35 pm on Jan 14, 2007 (gmt 0)

Whats the ip address its coming from?


 9:49 pm on Jan 14, 2007 (gmt 0)

My programmer isn't available at the moment, but when we checked the IP earlier we found that it was a real googlebot IP, not a fake or hacker.

We are not sure how long this has been going on, just caught it last night because our site went down, but since we restarted the server our programmer has been watching the logs and noticed what I explained in the initial post.


 10:10 pm on Jan 14, 2007 (gmt 0)

Go to this link:


and contact google. Follow the instructions. They might not respond until tommorrow though.


 10:17 pm on Jan 14, 2007 (gmt 0)

You also might want to deny the ip for now as well until its resolved.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Google / Google SEO News and Discussion
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved