This happened to my client over a period of two months. You might have experienced a random spike or, in the case of my client, we actually spent 2 hours on the phone with Google because we also saw a spike in the percentage of clicks Google deemed invalid.
Try checking to make sure you are not using the same phone number in the other Ad groups.
Check to see a change in your organic traffic.
click fraud is much easier than actually getting fake people to call as that would be expensive to run. If in doubt, call your Google rep and confront them about the issue