Spending at his department is running at €336m over budget and the HSE is heading towards a deficit of €400m by the end of the year.
"Others in Cabinet are already annoyed that he has missed these targets. Now there is all the extra information about what went on behind the scenes when the locations for the care centres were being picked," said a Fine Gael source.
"It will make it hard for him to facedown calls for extra cuts in his department when everybody sits around the table later in the week."
Minister Reilly's botched handling of his inquiry into the death of Savita Halappanavar has also weakened the political goodwill towards him from other ministers.
The source added that the Budget has to be very carefully crafted to make sure that there are no "over 70s-style medical card issues" buried in the detail.
"He's on thin ice with everybody at this stage -- the Government, the backbenchers and the public. You name it, he has upset them," the source told the Herald.
Over the weekend it was revealed that the Department of Health added four sites to the list of 35 locations for primary care sites on the evening before the list was announced in July.
It has also emerged that the top civil servant in the department launched an audit to see if there was any links between Dr Reilly and the primary-care sites in his constituency.
Locations in Balbriggan and Swords were among the late additions to the list.
The internal inquiry found that there had been "no impropriety" on the minister's part but Fianna Fail and Sinn Fein are trying to force the Dublin North TD out of office.
And former junior minister Roisin Shortall -- who resigned over the affair -- has said Dr Reilly's Dail explanation that his 15 additions to her original list of 20 were based on "logistical, logarithmic progression" was "absolute codswallop".
Fianna Fail leader Micheal Martin said today that other ministers have been concerned about James Reilly "for six months".
"Where is the Labour party in all of this and where is the Tanaiste Eamon Gilmore?" he asked.
Mr Martin described Minister Reilly's handling of the primary care centres as "shambolic".
"The minister was not telling us the truth, neither was the Government, in relation to how that process occurred."