Joe and his wife June have triplets, Sean, Ronan and Ellen, aged 17.
"There's a bus-stop near the house and it's only eight stops and you're in Malborough Street but I do worry about my kids being in town. There has been so many unprovoked attacks in the last few years," he explained.
The father-of-three has issued his children with strict instruction on how to deal with any trouble on the streets.
"I've told them if they're mugged to hand over their mobile phone, their handbag, or their laptop, that it's just not worth it. And I tell my lads, and I want other kids to take my advice, forget about what someone else has said to you, walk away, forget about it.
"Tomorrow is another day, because it's in these moments of madness in which whole lives are destroyed," he said.
However the trio have bigger issues on their minds heading into 2013 as they are all due to sit the Leaving Cert next summer.
"Bizarrely it helps that all three are doing it at the same time. If one of them was younger it could be a distraction. It's pretty intensive but I'm not putting them under any pressure. They all want to do their best."
The 51-year-old conceded that in the current climate, his three offspring, like so many Irish youngsters, may end up leaving the country
"I would be upset but that has to be on the agenda for all of them now. That option is unfortunately there. They talk about it: what they are going to do after their exams. Maybe a year in Australia or a studying thing. Hopefully they will continue to education and I would be right behind them on that," he said.
But the RTE star himself believes that 2013 will have to be a better year.
"2012, as far as the great unpleasantness that has been visited on us, had to have been the worst yet, in terms of the calls I got to my radio show.
"Last year people thought the Olympics or the Euro championships might help. They were wrong.
"It was a terrible year which is why people want to go into 2013 for a new start. Hopefully January 1 will see a new start."
Joe Duffy's RTE programme Local Heroes, celebrating individuals creating jobs in their local communities, returns to RTE One tonight at 8pm.