Welcome to WebmasterWorld Guest from 18.104.22.168
What was the general attitude, tone, reasoning? That should have given you a good feel for where you are.
My first guess it that they believe the charity link and the picture are being used in a context that could imply an endorsement. That could be a big problem. It makes me wonder that if the charity link were not there, and the picture were clearly captioned as, "This is my daughter with Fighter X.", if anything would have ever been said.
If it were me, I would take one of several courses:
1) Comply immediately.
2) Bring in an internet attorney immediately.
3) Comply within reason immediately, and show that you don't want to step on any toes, but that you are claiming that they are overstepping in the full extent of the demands.
4) Whatever you do - don't wait a moment.
Having an attorney in your back pocket is a good measure, but don't bring out the lawyer unless you are threatened with legal action.
IANAL (I am not a lawyer) but I don't think the celebrity has any rights concerning the photo with your daughter - if it was taken in public and with the boxer's permission - so they can't MAKE you take that down. But they can ask persuasively and if you want a good relationship with the celebrity (as a fan) then it'd be best if you do what they ask.
While you have them on the phone, ask if there IS any content they can provide which you can use on your fan site. Stuff no one else has. Can't hurt to ask.