There is actually a tool that estimates traffic to a URL. I can't find the address right now, maybe someone else has it?
As far as I can tell it's a good estimate, but it's by no means accurate. I had some fun checking my site against some of my competators but other than that I would not give too much creditability to their statements.
If you are concerned, ask the guy what numbers he has, then tell him "Ok, and what about our traffic the day before". Even if he knows how much traffic you get, shoudn't that give him something to worry about not you?
It could also be that he's using the Alexa info and thinking it's actual traffic. Who knows.
alexa toolbar data is a guess and therefore not reliable. Hitwise partners with isp's and reportedley can give you the best intelligence. It costs too much for me (around $5k for reports, that neighborhood).