The hosts had declared on 525 for eight shortly after tea.
Opening batsman Martin Guptill (1) survived Dale Steyn early on, after Jacques Kallis uncharacteristically put down a regulation slip catch in the first over. Spurred on, Steyn grabbed Guptill's wicket in his next over, as Alviro Petersen held on to a thick edge at third slip.
Kane Williamson took a few body blows from Morne Morkel before he too succumbed to Steyn, caught by Graeme Smith, for four. Williamson's pain was apparent as he walked gingerly off the field.
At 8/2, it was left to Brendon McCullum and Dean Brownlie to try and steady the ship. However, a snorter from Rory Kleinveldt, standing in for the injured Vernon Philander, saw off Brownlie, caught behind by AB de Villiers, for 10.
Proving worthy of his selection, Kleinveldt struck again, dismissing Daniel Flynn for a duck.
The wicket sparked the brass band into their rendition of the national anthem and brought the entire crowd to its feet.
And the drama was not over yet. Smith called on his spinner Robin Peterson to turn his arm and he immediately took two wickets in two balls. He saw off McCullum, with a beauty which turned out of the foot marks, and caught the edge of the bat to Kallis in the slips.
He went over the wicket with his next ball to left-handed debutant Colin Munro. The ball clipped the edge of the bat and looped gently to Dean Elgar at short-leg for Munro's golden duck.
The Black Caps survived another tense four overs and it was with great relief that BJ Watling and Doug Bracewell walked off, unbeaten on 15 and three, respectively.
Peterson's double-wicket maiden took the prize and he finished on 2/2 but all four bowlers deserved to share the honours with their outstanding bombardment of non-stop pressure balls.
Determination and reward
Faf du Plessis and Dean Elgar had piled on the runs after lunch with Du Plessis setting the tone, smashing sixes to bring up both his half-century and his second Test century in only his fourth match.
He was given one life, on 69, when the ball brushed his glove on the way through to the keeper but the umpire gave him not out.
His hundred came off 202 balls and included 12 fours and two sixes and he combined with Elgar for a 131-run partnership – South Africa's highest sixth-wicket stand against New Zealand at St George's Park.
Elgar, who made a pair in his first Test, and 21 in his only other innings – in the Cape Town Test last week – played with more determination, stroking some magnificent boundaries.
However, runs dried up after Du Plessis departed for 137, but captain Graeme Smith magnanimously delayed his declaration until after tea to allow Elgar his maiden hundred.
His unbeaten 103 came off 170 balls and included 14 fours and a six.
In a slow morning session, the Proteas lost one wicket after play got underway 15 minutes late due to some light rain. Lunch was adjusted accordingly after recently-updated International Cricket Council (ICC) regulations finally permitted lunch times to be movable.
The damp conditions made it difficult for both batsmen and bowlers and Hashim Amla, resuming on 106, added just four more runs before rewarding Trent Boult with his only wicket of the innings.
The Black Caps did well to restrict the Proteas to 65 runs in the first session of the day but after lunch they let it slip. Doug Bracewell was the pick of the bowlers bagging three for 94. Colin Munro took two for 40 while Trent Boult, Neil Wagner and Jeetan Patel took one wicket each. – Sapa.