Wales' Owen Williams tackles Japan's Harumichi Tachikawa during their international test match at Hanazono rugby ground
It was Japan's 12th straight defeat to Wales, including three losses in World Cup matches, since their first encounter in 1973.
"I'm hugely proud," said Wales coach Robin McBryde.
"We didn't deal well with the (hot) conditions in the first half, but showed really good character to come through," he said.
Full-back Ayumu Goromaru gave Japan a 6-0 lead with two penalties but Welsh stand-off Dan Biggar pulled two penalties back, one with a long shot from near the centre line, to level at 6-6.
Full-back Liam Williams saved a try, but the Japanese threatened the Welsh side several times and finally scored to take a 11-6 lead again shortly before the break.
Capitalised
Japan won a line-out close to the Welsh goal line, allowing right-flanker Michael Broadhurst to touch down near the right corner in the 38th minute, too far out for Goromaru to convert.
Goromaru missed two penalty shots while Biggar scored one penalty to trail Japan 9-11 at the beginning of the second half.
Wales encamped in the Japanese field for most of the second half, playing close to the Japanese try line and took the lead for the first time when Biggar added another penalty in the 59th minute.
The visitors stretched their lead when right-winger Harry Robinson completed a fluent passing play to score a converted try on the right field in the 63rd minute.
Right-winger Yoshikazu Fujita kept Japanese hopes alive with a try in the 69th minute and Goromaru capitalised to close the gap to 18-19.
But a Japanese high tackle gave Wales another penalty chance, which substitute Rhys Patchell cashed in on in the 78th minute to complete the scoring.
"I'm very disappointed with the game. We could have won. But I'm very proud of my players," Japan coach Eddie Jones said.
The two teams will play each other again next week at the Prince Chichibu stadium in Tokyo.
Wales' best players and their head coach Warren Gatland are touring Australia with the British and Irish Lions. – Sapa-AFP