The US-based star admitted that the term may have been too strong.
But he said that he ultimately stands by his sentiments.
The Usual Suspects actor attracted a lot of criticim when he lashed out at the event, which encourages Ireland's diaspora to return home in 2013, in a radio interview in November.
Byrne said in that interview that the Taoiseach's speech about the event was "slightly offensive".
"Most people don't give a s*** about the diaspora (in Ireland) except to shake them down for a few quid," he said late last year.
But, in a newspaper interview at the weekend, he said that, in hindsight, this criticism may have been slightly heavy-handed.
"Shakedown may have been too strong a word," he said. But I wouldn't take back anything that I said."
He said that the millions of Irish people living around the world are "more than tourists".
"I felt I was really well placed to say it because I have lived in America since 1987. I understand how complex that group of people is, the diaspora," he said.
"There were hundreds of thousands of people who left here in a few years.
"There are a lot of those people who are really angry, who did not want to leave. So they're saying, 'Why are we now being asked to go back when we were kicked out because of the mismanagement etc, etc?"
Byrne said that Irish people living abroad are tuned in to what is going on and would love nothing more than to go home.
"So there's a great deal of anger there, there's a lot of people who would love to come but can't afford it and there are people who say, 'You know what, we're sick of being described as tourists. We're emigrants, we're exiles, we have a real connection to that country."