Sri Lanka takes a 2-0 lead in the five-match series. Sri Lanka opted to bat after winning the toss at R Premadasa Stadium and scored a competitive 223-9 in 49.2 overs when rain interrupted play for the first time.
Captain Dinesh Chandimal and opener Tillakaratne Dilshan scored 43 runs each.
After a long stoppage, South Africa was given a revised winning target of 176 in 29 overs and managed only 104-5 in 21 overs when play was called off due to rain .
They were 17 runs short of the par score at that stage.
Herath dismissed Alviro Petersen, who top scored for the visitors with 24 runs, and had Faf du Plessis caught by wicketkeeper Kumar Sangakkara to finish with 2-16.
Sri Lanka's win came largely through team effort rather than any individual brilliance.
Chandimal, Dilshan and Sangakkara (37) accumulated the runs on a slow pitch to take Sri Lanka to a fighting score, while three bowlers picked up one wicket each.
South Africa went into bat without opener Hashim Amla who was injured trying to stop a ball during Sri Lanka's innings.
Robin Peterson (3), promoted to opener, was dismissed in the last delivery of the first over by a yorker from fast bowler Lasith Malinga.
A promising partnership of 32 runs for the second wicket followed between Petersen and JP Duminy, who edged a delivery from Thisara Perera to wicketkeeper Sangakkara while on 15.
Three more wickets fell for 30 runs before sixth wicket pair David Miller and Ryan McLaren shared an unbroken 35-run partnership in an effort to revive the innings, which proved futile as rain intervened.
Earlier, South African fast bowler Morne Morkel picked up 3-34 to limit Sri Lanka to a modest score.
Morkel removed both openers Upul Tharanga (3) and Dilshan, before later accounting for Chandimal.
South Africa started its bowling with the left-arm spin of Peterson and the first breakthrough came when Tharanga top edged a rising delivery from Morkel for a catch to Peterson at 7-1 in the fourth over.
Dilshan and Sangakkara, who hit a career-best 169 in the first match, shared a promising 59 runs for the second wicket before Sangakkara charged at left-arm spinner Aaron Phangiso only to find Petersen at wide mid-off.
Sangakkara faced 60 deliveries and hit six boundaries.
Mahela Jayawardene was bowled by Peterson trying to reverse-sweep for 17.
Dilshan, known for his aggression, hit just one boundary during a 64-ball 43 and Chandimal, who has struggled for runs in his last 11 one-dayers, showed signs of recovery with 43 off 51 deliveries.
The third match in the series is on Friday at Pallekele. – Sapa-AP