My step-father was actually in this situation with his daughter. Let me give you a little back story.

When he got divorced from his ex-wife, and before meeting my mother, it was the early 1980s (please don't judge this on the time period--I still see what I'm about to share happening to people I know and love today). He loved (still does love) his daughter very much, and from what little I've learned about his ex through my mother, she was anything but stable.

My step-father lost his job shortly after the divorce and had to either move home with his folks or do the homeless thing. He chose to move home. One weekend, he had his daughter over and she was in a baby walker. They were in a split level home, and they had baby gates up all over the place so she wouldn't take a tumble and hurt herself. Well, something happened with one of the gates, she bumped into it enough times, and she tumbled down about six steps in her baby walker. She was fine, but she did have a bruise or two.

That was the last weekend he was permitted to see his daughter on his own. His ex screamed child abuse and dragged him into court. Now, let me tell you something: If ANYONE has EVER given my step-father a reason to smack her upside the head, it was me as a teenager. I kid you not. I was a terror, I was a nightmare, and I was so mouthy I can't believe my mother didn't just smack my mouth off my face. After dealing with the mess that I was, there is just no way in my mind that he EVER could have done anything to that baby girl. But, she won her case anyway, and the only way he could see his daughter was if he went to her home and she supervised the visit (yeah, not a social worker, they weren't really prevalent back then). Of course she'd never be there or she'd refuse to answer the door. He ran out of money to fight her, and he lost the chance to have a relationship with his daughter out of this woman's horrible spite. In my opinion, she needed her head examined, among other things, but I was young then and knew nothing of the situation.

So what's my point? My point is that he may not have been playing with an ex who has a full deck (or a 1/2 a deck, even). Here's the thing: Without more details, it's hard to know if he's a loser or if he's been placed in an unfortunate situation and is resentful of that situation as a result. While I don't think he should feel like a glorified babysitter when he's with his child, if he's rarely permitted to be with his child, I can see how he could be angry enough at her to say that. No child or parent should be in that situation.

What should you do? Find out as much about the situation as you can. Is this an ex who is playing the system and a system that is allowing itself to be played, or if this a situation where you have a man who is apathetic and has the potential to continue to be so? If it's the latter, run, don't walk, to get away. However, if it's the prior, you have to do what is in your best interest and follow your gut from here on out. We don't always know the journeys people have had to take to get to where they are today, just that they are here. We don't always get to choose how it's going to go, either. Sometimes, the journey happens to us in spite of our best interests. Either way, I do think he needs some help dealing with his situation.

Best wishes to you, and sorry for the incredibly long response. I do hope it helps in some way.