Phiyega said she was not aware of the criminal investigations against Zuma.
"I became aware of the court charges against Major-General Zuma just after the media briefing today. I immediately met with him to establish the facts," Phiyega said in a statement.
"He confirmed that indeed he has appeared several times in court since 2008 when the matter was first heard. He will be appearing again next month, during September 2013."
Phiyega appointed Zuma as the Gauteng provincial commissioner earlier on Saturday in Pretoria.
Phiyega said Zuma had not informed the service of any pending criminal investigations against him.
She said in May, all South African Police Service workers were advised to disclose whether they had any pending criminal investigations against them. Failure to do so would be viewed as a serious misconduct.
"I have therefore taken a firm decision to permanently withdraw his appointment. Disciplinary steps will be taken against Major-General Zuma for his failure to declare the pending criminal charges," Phiyega said.
Divisional commissioner for visible policing, Lieutenant-General Lesetja Mothiba would be the acting provincial commissioner of Gauteng until further notice.
"In the time that I have known Major-General Zuma he has always come across as a decent, capable and committed police official and leader. I am sad and disappointed in him," Phiyega said.
Zuma has 20 years of experience as a policeman. He is not related to President Jacob Zuma.
He was meant to replace Lieutenant-General Mzwandile Petros whose last day as provincial commissioner was on Saturday.
"I wish to also take this opportunity to thank Lieutenant-General Petros for being a loyal servant of the service," Phiyega said earlier.
"He is the epitome of a professional police officer."
She said Petros had agreed to stay with the police for a year.
"We have been engaged in discussions about his future role in the police. He indicated his desire to move on and pursue other interests. I have tried hard to ensure that his skills are not lost to the SAPS. He has kindly agreed to stay on until the end of this year to focus on a number of special projects," she said.
– Sapa