Nah that would never work, not every RDM is from an arsehole, often its just plain accident. Also to repeat what Worbat said: admins really are the only answer. You cant take it into your own hands too often since it turns into a fuckfest, as ive had an argument about before.
What about a different idea, similar to NT, killing so many innocents as a non-traitor per map round, you would be punished by not being able to play the one round. I think it would seriously reduce the RDM without punishing mistakes too severely.