Davidson (59) was reportedly arrested at Heathrow Airport as he returned to the UK to be a contestant on Channel 5's 'Celebrity Big Brother'.
The station would not comment on his inclusion but he had been preparing to take part in the show, which returns to screens tonight.
He will now no longer be taking part and programme chiefs have been forced into a last minute rejig.
Scotland Yard said earlier that a 59-year-old man was held in west London at around noon yesterday after a 53-year-old man had been arrested in Hampshire, both on suspicion of sexual offences linked to Operation Yewtree.
They have since been released on bail until March, pending further inquiries. Six men and a woman were seen leaving Davidson's house in Hampshire yesterday carrying boxes.
The men, wearing smart trousers and jumpers, left the country cottage at about 7pm in three cars.
In a statement, Mr Brandman said: "Two women have made allegations in respect of Jim that date back approximately 25 years. The complainants were then in their mid-20s.
"Jim vigorously denies the allegations. He has not been charged with any offence."