Five penalties from centre Christian Lealiifano and scrumhalf Nic White proved just enough for the twice champions, who will now travel to Pretoria to play the Bulls next weekend.
The Cheetahs played some great rugby and scored the only tries of their first play-off match through centre Johann Sadie early in the first half and replacement winger Ryno Benjamin at the death.
Smit could have extended the Cheetah's fairytale season into 20 minutes of extra time but his kick from wide out on the left touchline came back off the near post.
Victory for the Brumbies means the Canterbury Crusaders will now play the defending champions Waikato Chiefs in Hamilton in the other semifinal next Saturday.
"You always worry when a South African guy's having a kick because they are usually dead on the money," said relieved Brumbies skipper Ben Mowen.
"Fortunately for us it fell our way. That's good to grab a win here but we're going to build now and try to run all the way through."
A huge battle at the breakdown between George Smith and Heinrich Brussow was just one of several intriguing contests that ensured the tightest of matches but ultimately three missed kicks from four attempts by Smit cost the Cheetahs victory.
The visitors took a 5-3 lead in the 16th minute when Willie le Roux's flat pass allowed fellow winger Raymond Rhule to breach the Brumbies defence and he off-loaded to give centre Sadie a clear run to the line.
Lealiifano missed his second and third penalty attempts into a tricky wind before nailing his fourth to put the Brumbies back in front 6-5 after half an hour but they came up empty handed after a long period inside the Cheetahs' 22.
Smit's first successful place-kick on this third attempt six minutes into the second half inched the visitors ahead but home scrumhalf White steered a long-range penalty over three minutes later.
Lealiifano then resumed kicking duties and slotted over two more three-pointers to give the Brumbies a seven-point cushion at 15-8 with 12 minutes remaining.
Desperate to keep their best season alive, the orange-shirted South Africans poured forward and laid siege to the home try line with replacement halfback Sarel Pretorius marshalling the attacks.
It was his pass that put Benjamin over in the corner with time running out to hand Smit the tough task of forcing extra time.
"Very disappointing to lose tonight but I'm very proud of the guys, we've had much better season that we can build on in the future," said Cheetahs captain Adriaan Strauss.
The Crusaders will go into the penultimate weekend of the competition as hot favourites for an eighth title after running riot to demolish the Queensland Reds 38-9 in the first play-off on Saturday. – Reuters