threat of violance

Tenancy Deposits

rented out a property to a married woman, tenancy was in her sole name,she lived with her husband in the property, i used the R.L.A tenancy agreemant, i took a deposit which i protected useing t.d.s.
Her husband phoned during the tenancy to say they were moving out in 2 weeks time during the period of the initial 6 month term, they left the property o.k. but as they only gave 2 weeks notice i retained equivelant of 2 weeks rent, no objection or dispute was raised at the time, bumped into the husband while out and about 5 months later and he threatend to beat me up and put me in hospital saying i hand ripped his wife off, this was in october2008 i reported it to the police they said they would go round and speak to him, bumped into him again today outside my business(shop) again he threatend me with violance and says if i do not give the 2 weeks rent back he will beat me up, i have an appointment at the police station for tomorrow to report it, this guy lives 400 yards up the road in another rented prop, would i have any redress with his current landlord over his threats? was i in the right to retain the 2 weeks rent as it was clearly a breach of contract, they did not then and have still not raised a dispute with Tenancy Deposit Solutions,what should i do, this guy is violant and i believe he will carry out his threats against me