"Downton was a very monopolising job. So there is a strange sense of liberation at the same time as great sadness because I am very, very fond of the show and always will be," he said.
Matthew died in a car accident just moments after kissing his newborn son and heir in touching scenes along with wife Lady Mary.
Matthew's was the second shock death in the third series of the period drama -- there was outcry when Lady Sybil died in childbirth earlier this year.
Stevens has gained legions of fans but there has been some speculation about his future on the show for the past number of months as rumours mounted that he had not signed a contract for the fourth series.
Dan has now been openly tipped to star in a film about Wikileaks' Julian Assange alongside Sherlock Holmes star Benedict Cumberbatch.
"I haven't done as many films as I would have liked. A lot of my contemporaries have done more. I don't have 'I will be a movie star' emblazoned on anything but I'd like to do a bit more screen stuff and then, when the time is right, come back to theatre.
"When it is good, theatre takes a lot of beating both to watch and perform."
Dan is currently getting rave reviews on Broadway where he is starring opposite Hollywood star Jessica Chastain (The Help) in The Heiress. "It is a desire for freedom really," he said when quizzed about quitting Downton.
SEE CHRISTMAS TV REVIEW ON PAGE 33