Crowds gathered at the 37 Military Hospital in the capital, Accra, in a mass outpouring of grief in the West African nation.
The news follows weeks of speculation about his health after a decline in public appearances and a trip to the United States for medical reasons, which the president insisted was a "routine check-up".
Mills's death was announced publicly on Tuesday afternoon after hours of rumours and a near communications blackout in Accra.
A statement from the office of the president said: "It is with a heavy heart that we announce the sudden and untimely death of the president of the republic of Ghana."
MPs were summoned to the Parliament in Accra for the swearing-in of the vice-president, John Dramani Mahama, amid reassurances that Mills's death would not spark a constitutional crisis.
But there were questions about how his death, the first of a serving president in Ghana's 55 years as an independent nation, would affect elections due to take place in December.
Mills, who had begun campaigning for re-election for the ruling National Democratic Congress party, was expected to fight a narrow contest against the opposition National Patriotic Party.
Earlier this month Mahama published a memoir, My First Coup d'Etat and Other True Stories from the Lost Decades of Africa.
It has been described as a coming-of-age account set during Ghana's post-independence years. – © Guardian News and Media Limited 2012