Well, then you are getting one for each new reply that that thread if getting. If you select to get notified for a high-traffic thread, it is likely you will be getting multiple notifications of replies.
Personally, if you want to keep tabs on a thread, either check it regularly or "flag" it and come back to it later.
To "flag it" there is an option down in the Thread Options section. You can then go back and look at it later if you want.
These two things are at least what I like to do. Otherwise you are going to have to live with multiple replies. Create a email filter to put them in a "folder" in your email; that way it won't be overwhelming.