He has young children, he told the President, and wants to move back to his home state of Georgia.
Trump pushed Ayers to commit to two years on the job, but he declined.
Ayers revealed the news in a tweet.
"I will be departing at the end of the year but will work with the #MAGA team to advance the cause", he said.
That leaves it unclear who will take the top West Wing job as Trump enters the 2020 campaign confronting growing danger from the special counsel investigation into Russian election interference and from Democrats who have vowed tougher oversight after they take control of the House next month.
The official says that Ayers will instead assist the president from outside the administration.
Also emerging as a candidate is Rep. Mark Meadows, a North Carolina Republican and the chairman of the House Freedom Caucus. Prior to his service in the White House, Kelly served as a general in the Marine Corps.
Mr. Trump and Ayers, as CBS News has reported, had been negotiating possible terms for the job.
"I think Donald Trump desperately needs in that job someone who can walk into the Oval Office, close the door, and tell him what he does not want to hear", Whipple said.