Sand - 9:57 pm on Jun 10, 2012 (gmt 0)
I have about 6 sites that I actively work on, and only one has been hit by anything (the April Panda updates and whatever happened this past Friday). The rest have continued to grow at a steady clip. None of these are anything close to being a brand. All could be described as niche sites.
All of my sites do have many high quality links from some of the strongest domains out there. That's actually one of the criteria I take into account before starting a site -- identifying authority link targets. If I can't find many, I know I'm not taking the right direction and need to either pivot on the idea or scrap it altogether.
None of these sites have any spammy links beyond the ones scrapers create. All have original content and have content that gets shared.
I don't have all the answers (or many at all), but I have learned the hard way that you can paint yourself into a corner by going too niche if there's no active audience that's dying to talk about the topic you're chosen.