At next committee meeting put all the cards on the table (that you have proof of not what you have heard second hand or by inuendo), ask the committee how they believe things should be resolved as things obviously aren’t working as they are.
With other members of the committee involved invite him to a meeting to address what ever issues he may have, make a structure for the meeting giving both sides opportunity to state their issues. Once these have been raised agree a time for a second meeting so that each side can prepare correct reponses to each accusation/problem that has risen (make sure this structure is known by all concerned before starting).
If its as you say and he is in the wrong the committee should back you and either a strongly worded letter signed by all committee members warning of his future behaviour should be sent, or an outright ban could be imposed depending on the outcome of the meetings.
At each meeting make sure there are independant observers making notes/taking minutes that should be agreed on by both parties.
If the above doesn’t work get the bombers out and wee in his shoes……..