Welcome to WebmasterWorld Guest from 54.226.241.8

Message Too Old, No Replies

Googlebot is crashing my site

googlebot is on an infinite loop of one of my pages, help!

     

eddytom

4:46 pm on Jan 14, 2007 (gmt 0)

10+ Year Member



I run a site that has over 2000 pages. Normally we are crawled thoroughly and wonderfully everyday. We have five sections of archives pages. Last night googlebot got stuck on the first page of each of our five entry archive pages. Googlebot is stuck on an infinite loop of crawling those pages and we can't stop it. It is not going anywhere else (other than the homepage, which it goes to every 5 minutes) on the site, just those 5 pages and the homepage. We put a meta name='robots' content='noindex,nofollow' tag, it didn't stop it, we put a meta name='googlebot' content='noindex, nofollow', it is still stuck and we don't know what to do. It is not following any of the links on any of our pages and the constant loop is causing us to lose memory and we keep getting shut down. Any idea what the heck is going on?

trinorthlighting

9:35 pm on Jan 14, 2007 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



Whats the ip address its coming from?

eddytom

9:49 pm on Jan 14, 2007 (gmt 0)

10+ Year Member



My programmer isn't available at the moment, but when we checked the IP earlier we found that it was a real googlebot IP, not a fake or hacker.

We are not sure how long this has been going on, just caught it last night because our site went down, but since we restarted the server our programmer has been watching the logs and noticed what I explained in the initial post.

trinorthlighting

10:10 pm on Jan 14, 2007 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



Go to this link:

[google.com...]

and contact google. Follow the instructions. They might not respond until tommorrow though.

trinorthlighting

10:17 pm on Jan 14, 2007 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



You also might want to deny the ip for now as well until its resolved.
 

Featured Threads

Hot Threads This Week

Hot Threads This Month