The INLA has disbanded and decommissioned its weapons stating that the "war is over", but criminal elements remain and some view Duffy as a traitor.
While living in Dublin in 2007 and 2008, Duffy was involved in spats with drug gangs, and had death threats.
But he was sacked as head of the INLA after he was arrested and charged with membership of an illegal organisation, for which he was later jailed.
The 36-year-old received a separate 24-year sentence in Britain last week for the 1992 killing of Sergeant Michael Newman but is likely to benefit from the early release scheme that was set up by the Good Friday Agreement.
A security source told the Herald: "Duffy is unlikely to ever return to Dublin. Threats have been made against him, some from former comrades. He is likely to relocate to Co Armagh on release."
The INLA threatened to release a statement saying Duffy was never a member so that he would have to serve his full sentence but no statement has been issued.
The former terrorist leader became a figure of derision within INLA divisions after he publicly disassociated himself with the organisation in 2009.
Duffy was beaten in prison a year ago after his statement and a prison officer was also injured as he tried to come to Duffy's aid.
He is likely to be released in just two years' time, in accordance with the provisions in the peace document.
Sgt Newman, a father of one who never served in Northern Ireland, was out of uniform and unarmed when he was gunned down in a car park in the middle of Derby in broad daylight.
An associate of Duffy, Joseph McGee, has already been convicted for his part in the murder of Sgt Newman and sentenced to 25 years but served only two years before he was released under the terms of the Good Friday Agreement.
McGee was arrested in 2004 when he crossed the border to attend a funeral in Co Armagh, and later pleaded guilty to the murder. A third man, Patrick Gorman, was arrested last year and is challenging his extradition to Britain.
Declan Duffy told Stafford crown Court last week that he regretted the killing and for him "the war is over".
"My war is over and there are things I have to get off my chest," he said. "This man was a family man and it is regrettable that he was killed.
"I would be happy to meet with any member of his family to explain to them the circumstances of why soldiers at the time were being targeted.
"The war is now over and I acknowledge the hurt caused to Irish and English people."