They did not say how this would affect a plan for regional military intervention in northern Mali to drive out Islamists and Tuareg separatists, who seized two-thirds of the country in the chaos after a March coup.
Although the soldiers gave way to a civilian president and prime minister in April under international pressure, they have never been far from power.
Prime Minister Cheick Modibo Diarra resigned and dismissed his entire government on Tuesday after being arrested and taken to a barracks that served as headquarters for the former junta.
The head of the AU Commission, Nkosazana Dlamini-Zuma, "firmly condemned" the conditions of Diarra's resignation and repeated the demand "for the total subordination of the army and security forces to the civilian government".
Sweating before the cameras, Diarra announced his resignation on Tuesday in a live 4am broadcast on state television. The 60-year-old former astrophysicist had been arrested hours earlier.
France called for a new government to be formed quickly and said the turn of events enhanced the case for foreign military intervention. But observers warned that plans for a United Nations-backed force to combat Islamist insurgents in northern Mali were now "on ice".
Diarra was preparing to leave for France when soldiers reportedly smashed down the door of his home, bundled him into a car and drove to the Kati military base where a coup had been launched on March 21.
Hours later, he addressed the nation with an apology. "Our country is living through a period of crisis," he said. "Men and women who are worried about the future of our nation are hoping for peace. It's for this reason that I, Cheikh Modibo Diarra, am resigning along with my entire government on this day, Tuesday, 11 December 2012. I apologise before the entire population of Mali."
For several weeks, tension has been mounting between the officers who led the coup and the civilian prime minister they were forced to appoint when they handed power back to a transitional government.
Bakary Mariko, a spokesperson for the junta, said Diarra was forced out because he was "not getting along" with interim president Dioncounda Traoré or coup leader Captain Amadou Haya Sanogo.
"It's the reason why Mali's army has taken things into their own hands and told Cheikh Modibo Diarra to resign for the good of Mali," Mariko told the Associated Press.
"For several days now, Cheikh Modibo Diarra has mobilised his supporters and boycotted the national conference [currently being held to discuss Mali's future]. Now he says he's going to Paris for medical tests ... but we know better and realise that he is trying to flee in order to create a blockage in the Mali situation."
Mariko told Reuters: "This is not a coup. The president is still in place, but the prime minister was no longer working in the interests of the country."
A foreign diplomat, who did not wish to be named, said Diarra's medical tests claim rang true. "Enough people have told me he was genuinely ill."
The Mali capital, Bamako, appeared calm, but the exit illustrated how the junta still calls the shots, even though the soldiers made a show in April of handing power back to civilians.
Last weekend, Diarra helped to organise a demonstration calling for a UN-backed military intervention in north Mali, which fell to Islamist militants in the wake of the coup and has provided a haven for al-Qaeda affiliates.
On Monday, European Union foreign ministers approved training for a mission involving Malian and other African troops.
In response to Diarra's resignation, Philippe Lalliot, a French foreign ministry spokesperson, said: "These developments underline the need for the rapid deployment of an African stabilisation force. All the things we've talked about are on ice, at least. Everyone's waiting to see what happens next."
This view was echoed by Germany's foreign minister, Guido Westerwelle. "One thing is clear: our offers of help come with the condition that the process of restoring constitutional order in Mali be conducted credibly ... Interim president [Dioncounda] Traore and all the country's political leaders must now act responsibly so that Mali returns to stability."
But there are doubts about whether he can assert his authority. In May Yerewoloton, a violent movement believed to be backed by the junta, broke into the presidential palace and severely beat Traoré.