drall - 5:00 pm on Sep 20, 2010 (gmt 0)

Tedster, it's very sophisticated.

How? First lets look at the data.

- PR 8 site.
- 1.1 million searchs per month.
- 120,000 pages handbuilt over 12 years.
- 36,000ish searchs per day

Up until 2005 this sites traffic would flux with the workweek. So lets say it looked like this.

- Sunday 50,000
- Monday 29,000
- Tuesday 31,000
- Wednesday 30,000
- Thursday 35,000
- Friday 39,000
- Saturday 47,000

Daily deviation would be as extreme as 50% sometimes. Totally natural. Then something happened slowly over time.

Until the week started looking like this and has stayed this way since.

(last weeks actual log)
- Sunday 36,425
- Monday 36,527
- Tuesday 36,487
- Wednesday 36,427
- Thursday 36,430
- Friday 36,428
- Saturday 36,454

If we have a large amount of new IBL's come in, nothing changes.
If we pump out a new section with tons of content, nothing changes.
If we remove large amount of content, nothing changes.
If we do anything, nothing changes.

I was convinced we had been hacked at one point but our security team shows an all clear.

Now the how part.

As with any good webmaster I have understood the power of a raw log file from the beginning I know what I am looking at and it is simply jaw dropping. The level of computational power Google must use to throttle boggles my mind.

Entire sets of randomly rotating serps that are tailored throughout the day to throttle us. Lately as I have been pointing out in other threads pages with social networking strength have been rocking for us.

So much so that some of these pages can bring in 3-4000 visitors a day. What is incredible to watch is after our visit count spikes for some page like that, other pages that normally perform #*$!x visitors per day will all suffer just enough the rest of the day to level us out to that 36,000 number.

My gut tells me that our root domain has been flagged somehow. A daily limit was assigned after team review. That root domain is "in the system" and balanced via tracking throughout the day.

My hunch is that it is a reverse engineered adwords/doubleclick for publishers daily budget planner but budgeting traffic instead of clicks or impressions.

Basically our domain is the campaign!