However, he had the least number of votes from the judging panel.
A smiling Rylan thanked the public before leaving the stage carrying mentor Nicole Scherzinger over his shoulder.
Meanwhile, a seemingly demure RiRi took to the stage in a floor-length black lace dress for the performance of her latest single, Diamonds.
But when the spotlights came on, they revealed a sensual Rihanna in a see-through lacy number.
US singer Bruno Mars also performed on last night's show.
Rylan had survived weeks of risky performances that often left Gary Barlow less than impressed.
All of the judges -- apart from Rylan's mentor Nicole Scherzinger -- opted to save Union J and send the controversial performer home.
But this was not before praising Rylan for his entertaining performances and upbeat spirit.
His final performance during the quarter-final sing-off even won praise from Gary Barlow, who described his rendition of Athlete hit Wires as "your best performance of the series".
"You have been an amazing contestant. We've had great fun, and none of it has been personal," Gary told him.
Louis Walsh called Rylan "fantastic" and fellow judge Tulisa added: "You've entertained me throughout."
Good-humoured Rylan was delighted to have been part of the show: "The second I was in the bottom two with Union J, I knew I was going home."
"I've had seven bonus weeks as far as I'm concerned and what a great time to go out at the quarter-final."
Union J, who sang Snow Patrol's Run in the sing-off, will join Jahmene Douglas (20), James Arthur (24) and Christopher Maloney (34) in Saturday's semi-final.