That was rude of him. He should have asked before defacing YOUR music.
In that situation I'd have said (loud enough for everyone to hear), something like 'HEY, that's MY music, mark your own when you bring it, NOT mine!!
Then I'd have taken my music back off him & said if that's how you treat me I'm not sharing with you any more, find someone else!
Is he assuming your music is now his? Or is he assuming a superior status over & above you, a sort of 'what's yours is mine' entitled kind of attitude.
If you can't or don't want to confront him yourself I suggest making a strong complaint to the leader of that group.