Baloyi's decision on Tuesday follows three days of violent protest against the planned move.
"People were under the impression that this [the merger] was going to happen, that it was already decided when it wasn't," Baloyi told journalists at KaraboFM – a community radio station in Sasolburg.
The minister said he would appoint a task team to investigate the viability of the proposed merger.
"At this point in time, there has been no merger of these two municipalities and there won't be any such merger until the task team has completed their work," he said.
Residents have opposed the proposed merger of the Metsimaholo municipality in Sasolburg with the Ngwathe municipality near Parys.
They claim Ngwathe is badly run by allies of Free State Premier Ace Magashule and is beset by nepotism and corruption.
"The Ngwathe municipality has run itself into the ground and we as residents of Metsimaholo do not want to be associated with those thieves," Zamdela resident and protester Sam Mthembu told the Mail & Guardian.
Protesters – primarily from Zamdela informal settlement – barricaded streets, torched vehicles and government property and pelted bystanders with stones in a show of opposition to the planned move.
Shops and businesses closed their doors, schools suspended classes and the local Sasol plant suspended operations indefinitely in response to the protests.
Police clashed with protesters in running battles on the streets of Zamdela, firing rubber bullets and teargas in an attempt to disperse the demonstrators.
Several protesters were seriously injured during the clashes with unconfirmed reports of some being killed.
Police would not confirm the death of any protesters late on Tuesday.
Baloyi said that while Zamdela residents were fully within their rights to voice their concerns against the proposed municipal merger, he was disappointed protests turned violent.
"Clearly there are elements of criminality to the Zamdela protests, but we will deal with that once the task team has finished its work," he said.
Meanwhile, the ANC on Tuesday condemned the looting and violence during a protest by Zamdela residents.
"Our utmost concern is the level of criminal activity taking place, including looting of shops," spokesperson Jackson Mthembu said.
"This will not only affect business in the area, but will also affect innocent residents."
Protesters on Tuesday tried to burn down the local police station and set a car and shop alight. Police had to fire rubber bullets for the second day running.
On Monday, protesters set two trucks alight and injured two police officers inside a nyala, which had become stuck in mud.
Water cannons and rubber bullets were used to disperse the crowd. Some residents barricaded roads and threw stones at police cars.
Police said 187 people had been arrested since Sunday in the protests.
The ANC praised government and law enforcement agencies for the "swift intervention" and the arrests.
"This cannot be allowed in a modern democracy such as ours, where individuals with ulterior motives take … what should be a peaceful protest to be a fertile ground for criminality," Mthembu said. – Additional reporting by Sapa