I have experimented quite a lot with both new and established authority sites, and I came to believe that once delta_n/N ratio hits a certain threshold your site will go down in serps, at least temporarily.
N is the total number of backlinks an delta_n is your links acquisition rate (number of new links per week, month, etc).
Of course, this is a very simplified description, and I don't have a large enough sample size of websites and links to claim that's how it always works. But that's how I think about it, and in my experience it's pretty close to reality.
Moreover, the same idea applies to specific keyword ranking. If you acquire too many links of some specific anchor text, comparing to the overall number of links with THIS anchor text + SYNONYMS that you already have, your site will temporarily go down in serps for the corresponding keyword query.
I have seen it more than once. Build a histogram of your incoming link anchors. If by chance you have a keyword you are ranking on 1-2 page for, but no (or almost no) links with the corresponding anchor text, try to add a few links with this anchor text and see what happens. In similar cases. my ranking temporarily go down for that keyword and bounce back in a few months.