The president revealed his longtime White House foe had been asked to resign as the government's principal lawyer in a tweet that also announced Session's Chief of Staff Matthew Whitaker as Acting Attorney General.
"At your request, I am submitting my resignation", Sessions wrote.
The Justice Department has the ultimate authority over Mueller's investigation into Russia's alleged interference in the 2016 presidential election and interactions between Trump associates and Russian officials.
But Trump immediately tweeted that former US attorney Matthew G. Whitaker, chief of staff to Sessions, will serve as acting attorney general.
"Deputy Attorney General Rod Rosenstein's letter appointing special counsel Robert Mueller does not give Mueller broad, far-reaching powers in this investigation", the erstwhile AG chief of staff wrote.
Democratic Representative Jerry Nadler, who will likely become chairman of the House Judiciary Committee - which has congressional oversight over the Mueller probe - immediately responded to Sessions' resignation by demanding an explanation for the change.
Senator Lindsey Graham, a Republican representing SC, vowed to help President Trump find a worthy successor for Sessions.
Senator Susan Collins said: "It is imperative that the Administration not impede the Mueller investigation". "And that attorney general doesn't fire Bob Mueller, but he just reduces his budget so low that his investigation grinds to nearly a halt".
Trump: President Donald Trump tweeted that Matt Whitaker will take over as acting attorney general.
Senate Minority Leader Chuck Schumer (D-N.Y.) on Wednesday called on Whitaker to recuse himself from the Mueller probe, citing his previous remarks on the matter.
We hope that the next Attorney General will address overly harsh prison sentences and work with the White House and Congress to reduce mass incarceration at the federal level.
Sessions had recused himself from special counsel Robert Mueller's probe into possible coordination between Trump's presidential campaign and Russian Federation.
Mr Trump, a long-time critic of Mr Sessions, has thanked him for his service. "But when everybody sees what's going on in the Justice Department - I always put "justice" now with quotes".
"I am committed to leading a fair Department with the highest ethical standards, that upholds the rule of law, and seeks justice for all Americans", Whitaker said.
Mr Sessions was one of Mr Trump's earliest supporters.