Gervinho of Arsenal crosses the ball as Pablo Contreras of Olympiacos closes in during the Champions match between Arsenal FC and Olympiakos FC.
Gervinho's 43rd minute-opener was quickly canceled out as Kostas Mitroglou headed the Greek champions level before halftime, but Arsenal was back in control after the break to ensure it stayed top of Group B.
Gervinho turned provider in the 56th, picking out Podolski in the penalty area and the German striker's shot went straight through the legs of goalkeeper Balazs Megyeri into the net.
Ramsey made sure of the victory in stoppage time.
"The second half was better, in the first half we were sloppy sometimes with the passing, we lost a lot of ball," Arsenal captain Thomas Vermaelen said. "We started well but the first half was not great."
While Olympiakos is yet to earn a point at the bottom of Group B, Arsenal has a maximum six points after beating Montpellier 2-1 two weeks ago.
"It gives us more confidence to have two wins from two," Vermaelen said. "The most important thing is to win your home games. We want to keep it going."
But the Gunners were hesitant in the first half in the wake of their first loss of the season on Saturday in the Premier League against Chelsea.
"It probably affected our confidence a little bit getting [beaten] … maybe there was a little bit of a carry-over," said Arsenal assistant coach Steve Bould, who spoke as manager Arsene Wenger served a touchline ban.
'Back on track'
"But we're hoping now that we've won, that has gone, done and dusted, and we get back on track."
Arsenal had initially pressed with conviction and Santi Cazorla's free kick was heading into the net until Megyeri produced a fingertip save.
Megyeri's opposite number Vito Mannone had to be alert, though, to block a low strike from Mitroglou from the edge of the area.
An edginess has returned to Arsenal's defence after a solid start to the season and it looked like being exploited again.
A cross from Giannis Maniatis was allowed all too easily to find its way into the penalty area. Arsenal was only rescued by Paulo Machado's poor finishing, with a scuffed connection and the resulting shot scooped over with the goal at his mercy.
"We started well, with a high tempo and that was great but then we dropped a little," Vermaelen said.
Olympiakos' defence was breached first as Gibbs and Podolski combined down the left flank and the ball reached the edge of the penalty area where Gervinho seized control and struck it into the net.
The visitors immediately pushed for the equaliser but Machado was too rushed as he shot from distance and Mannone easily collected the shot.
'A little bit nervy'
The equaliser came a minute later when Lenadro Greco sent with precision from the right flank into the penalty area, where Mitroglou rose between the centre backs to nod a header into the net.
"We looked a little bit nervy," Bould said.
And Arsenal's first chance to regain the lead in the second half was squandered when Cazorla missed the target after Gervinho did the hard work, cutting the ball back to his unmarked teammate in front of goal.
But after Gervinho's interplay with Cazorla on the right, Podolski put Arsenal in front, with a shot through the legs of Megyeri.
"Today he had two or three good saves and when it comes to the goal, in football you can always only do the best you can," Olympiakos coach Leonardo Jardim said through a translator.
Mannone started the move that led to Arsenal's third, with a long throw flicked on by Olivier Giroud to fellow substitute Ramsey. The midfielder went past captain Giannis Maniatis and coolly lifted the ball over the onrushing Megyeri.
"The result doesn't represent Olympiakos … but this result has given us a lesson that in this competition it's very important to take your chances," Jardim said.
Inspirational goalkeeping
Manchester City required a 90th-minute penalty from Mario Balotelli and some inspirational goalkeeping by Joe Hart to escape with a 1-1 draw against Borussia Dortmund in the Champions League.
City was outplayed for long periods by the German champions but somehow emerged with a first point in Group D when Balotelli rolled in a spot kick after Dortmund defender Neven Subotic handled a hooked shot by Sergio Aguero.
Marco Reus put Dortmund deservedly ahead in the 61st minute but despite a flurry of chances in both halves, the German champions were denied time and again by a string of saves by Hart.
"We didn't play well. We didn't deserve to take this point but in the end it could be important," City manager Roberto Mancini said. "Joe Hart saved us from an incredible defeat because he saved everything and we should say thank you to him."
In losing at Real Madrid in its first game, City had already put itself under pressure in what is widely seen as the competition's toughest group.
A second straight group-stage exit will be unacceptable to City's Abu Dhabi owners, who have poured money into the club since 2008. Yet this result leaves the English champions with a mountain to climb, even at this early stage.
Out of sight
They are already five points behind Madrid, a 4-1 winner at Ajax on Wednesday and three behind Dortmund.
If it wasn't for Hart, Dortmund would have been out of sight by the time Balotelli converted his penalty, nine minutes after the Italy striker came on as City's final substitute.
"We've got to be happy with a point," Hart said. "We kept ourselves alive in the group."
In an open first half that was end-to-end and played in pouring rain, the England international tipped shots from the lively Mario Goetze onto the post and crossbar and also saved brilliantly from Ilkay Gundogan.
Hart also produced a number of great blocks early in the second half, denying Goetze and Gundogan, but he was unable to stop Reus.
The nimble Germany forward pounced on a sloppy pass by substitute Jack Rodwell across the defensive line and raced through before placing a shot to the right of Hart, who got a hand to the ball but couldn't stop it sailing into the corner.
Enterprising build-up
Reus' goal meant City has conceded in every match this season, the home defence having no answer to the enterprising build-up play of Dortmund's technically gifted forwards all match.
"I was really happy with my team's performance, I was really proud of them," Dortmund coach Juergen Klopp said.
"I don't know when City last lost at home and we came really close. When you concede a goal so near the finish, it is hard to decide if you are happy with the point."
City, whose fans were outsung by Dortmund's raucous away support, had chances of its own, mostly because of the silky skills of David Silva.
The Spain playmaker set up Argentina striker Sergio Aguero for two one-on-one opportunities that were saved in the first half, while right back Pablo Zabaleta blazed over with the goal at his mercy after running onto another slick pass from Silva.
City was under constant pressure for most of the second half, though, but emerged with a point thanks to some harsh refereeing and Balotelli's coolness under pressure.
Subotic was unhappy with the awarding of the penalty, after his outstretched right arm blocked the ball as Aguero turned it goalwards.
"Never, never, not a chance," the Croatia defender said. "I have a hand, I can't cut it off. It was never in my life a penalty." – Sapa-AP