South Africa-born Pietersen quit white ball internationals after team management refused him his wish to appear in Twenty20 matches while opting out of 50-over games.
"It's impossible to know what you're giving up, and how hard it will hit you, until it's gone," Smith, who has had plenty of run-ins with Pietersen in the past, told online magazine Wisden EXTRA on Monday.
"One-day cricket has been a big part of his life for a very long time and, right now, he thinks he can just walk away from it … Maybe he can. But I think he'll quickly find there's a big hole in his life which isn't as easy to cope with as he may think.
Pietersen suggested last week he would be willing to come back to one-dayers if something could be done about making sure he wasn't burnt out by England's packed schedule.
But the England and Wales Cricket Board (ECB) are holding firm to their stance that players cannot "pick and choose" which limited overs matches they play in.
But Pietersen is still set to feature in this week's first Test against South Africa at the Oval.
Smith added: "I don't have any inside information at all, so I'm just looking from the outside, but if there's any truth that his decision was partly based on the disagreements he had with the ECB and the way they treated him, then I believe he'll find it even harder to come to terms with the way his one-day career has ended.
"He just seemed to be back to his aggressive, attacking best, too. Very strange that he should give it away."
The best is ahead
Smith gave up the South African one-day captaincy after last year's World Cup, but has no plans to follow Pietersen into limited overs retirement.
"He's just turned 32, only a bit older than me, but I reckon my best ODI form is ahead of me, not behind me. I'm very keen to play limited-overs cricket and I'm still very ambitious."
Although Smith's two previous tours of England, have resulted in the resignation of opposing captains Nasser Hussain and Michael Vaughan, the opening batsman is not expecting to add Andrew Strauss to that list.
"I could sense the pressure building on Strauss before the West Indies tour (of England) and I couldn't help putting two and two together," said Smith.
"What if they had a bad series against the Windies and he didn't score many runs? But I'm pretty sure those two hundreds have made him safe now … He [Strauss] is obviously a very strong personality with that quiet determination that sees him getting things done and achieving things with the minimum of fuss. He seems to keep an iron fist in a velvet glove."
South Africa, the last visiting side to win a Test series in England when they triumphed 2-1 four years ago, will replace England at No. 1 in the International Cricket Council Test rankings if they win this three-match series.
"England are more disciplined, smarter and better prepared than four years ago, and all the players who are still around from the last series have got better," said Smith.
"They are a very, very good team at home and deserve to be ranked No. 1."
But he added: But we have also got better and we're also pretty useful in English conditions. They have home advantage and they're ranked ahead of us, but I'm not going to compete for underdog status!
"If some people want to make us favourites then I'll take that as a compliment, not reject it." – Sapa