Sorry I didn't reply sooner, but I wanted to see what happened after a month. So after I posted this I was at 6.5 Million links. Now I'm at 7.4 million links. I didn't get the two million like in november, almost a million.
So maybe that was a fluke. I got the scared of my life though around Christmas Eve and Christmas. Lost over 30-40% of traffic. And then continued two days, after that.
After looking at my logs, I think mod_security was blocking like a ton of people. After turning it off, traffic seems to be up again. We'll see though. I just switched host also on the 23rd.
If anyone is interested, I can update this thread every month or every few months. Not sure if anyone has done a thread where they get millions of links. I update everyday pretty much also on this particular website. And like I said in my first post I might be an "authoritative" site.
I can make this into a case study, but I'm not really sure what to look for or explain. I was never effected by panda also.
Edit. Also traffic appears to be climbing still *knocks on wood*