Keyword stuffing as it literally means is stuffing your page content with the same term i.e. ABC in page title, meta description, content, internal linking etc. In short, an overdose of the same keyword in such a way that it does not seem natural or for the benefit of reader.
I think your question is different here. Keywords/anchor texts linked to a domain does not mean keyword stuffing. I think what you need is diversification in anchor text used for linking to other domains as well in domains that you link to.
Hope this helps.