Spokesperson Craig Lambinon said Albertus Myburgh (41), Wouter van Heerden (25) and Rodney van der Mecht (35) had been motoring close to Noordhoek on Monday evening.
They swam back to the boat after it capsized under the weight of the whale.
Lambinon said one of the men's cellphones was still working and they called the boat's owner for help.
"On arrival on-scene a search commenced for the boat in dark, moonless conditions, and using illuminating flares to aid in the search, the boat was found about one nautical mile off-shore with all three men clinging onto the hull," he said.
Myburgh had broken ribs and an injured arm and leg. He was taken to hospital in a serious, but stable condition.
Van Heerden's arm was injured and Van der Mecht sustained minor injuries.
Lambinon said the boat was towed to the Noordhoek ski-boat club. The centre console and pontoons were found to have been extensively damaged.
"The species of the whale is unknown to the three men, but they say it was large."
A pod of Humpback whales was recently seen in the area. – Sapa