renoirm - 3:50 pm on May 3, 2010 (gmt 0)

Good to see this post is live.

I run a pretty decent size website (600k uniques) and we started seeing this problem on one on of our subdomains on April 19. It had an outage due to high amount of http calls from googlebot. We corrected issue but saw about a 15% decrease in traffic only on the subdomain. Then on Apr 29th bam....down 20% on main site (high amounts of calls via googlebot) then down another 50%.

We run chartbeat which allows for real time visualization of the traffic and literally at around 10pm est the traffic began to drop at a 45 degree angle for 3 hours.

Then on May 1st at around 6pm EST we saw all of our traffic return. We were relieved until yesterday morning noticed the traffic was back at -70%.

The sad thing is since Jan 1 we've been rising at about 5% a week and were hoping "skys the limit" but then this hit.

I am pretty active in tech community in NYC and called a few of my friends who have websites. 2 of the people run websites with names you'd know. All in the long tail niche. PR6-PR7. All have said they saw a drop ranging from 15% - 30%.

I assume this is temporary due to the fact of Sataurdays return for 6 hours of traffic but if it's not it'll mean for plenty of work for my team and I.

We've read thru all the theories and the things we've seen consistently is:

1. Began around Mid April - May 1

2. Seems targeted at midrange sites (200k - 2mil)

3. Googlebot is giving sites a bit too much love

4. Numbers range in decreases (15% - 90%)

5. Doesn't appear on all data centers (Singapore traffic is great..even though I am an American Company)

Oh and 1 final thought and proof it's going on. In Google analytics I went to Benchmarking. Compared our site to various other sites. All have a unique user outage. Now I know the Benchmark isn't the best tool but it's interesting to see comparable in various niches with traffic drops. (all the drops I see are on April 28th.)