Reid previously announced he was leaving after two seasons. Demi Lovato appears likely to return, joining Cowell on the revamped panel.
The departures are yet another shake-up for X Factor, Cowell's attempt to strike gold in the US.
Cowell replaced panellists Paula Abdul and Nicole Scherzinger and host Steve Jones after a disappointing first season, and ratings dropped in the second season.
The December season finale drew 9.7m viewers, compared to more than 12m for the first year's season-ender.
Although the series has failed to match Cowell's bold pre-debut predictions, and certainly hasn't eclipsed sister Fox show American Idol, it has helped to shore up the network's autumn schedule.
Pop star Spears, however, proved less than a one-season wonder.
Combining fame and a history of erratic behaviour, she was seen as a potential spark for better ratings.
But reviewers criticised her for appearing inexpressive to the point of boredom and for a repetitively dull use of the word 'awesome'.