It made it four winning visits in a row to Hillsborough as ten-man Rotherham toppled the current Championship leaders.
Dutch winger Etienne Esajas scored twice, but then made the vital miss with his penalty as his night ended in despair.
Esajas thought he had produced a winner three minutes from the end of extra time but Reuben Reid levelled a minute later to signal the shoot-out.
With five minutes gone, Deon Burton pounced on Lee Grant's free-kick as it dropped kindly in the area and snapped off a shot that ricocheted out for a corner.
And when Steve Watson collected a ball on the edge of the box his deft, curling effort had Andy Warrington stretching but it stuck the angle of post and bar.
A Leon Clarke shot drifted wide after good work by Jimmy Smith before 90 seconds of quick-fire action warmed up the crowd.
Wednesday winger Esajas smashed a left-foot shot that deflected off Mark Lynch and Warrington dived at full stretch to parry it away.
Almost immediately, Burton collected the ball wide on the right and drove an astute angled pass that wrong footed the Millers and this time Esajas arrowed his shot just inside the post after 14 minutes.
Wednesday fans were still celebrating the goal when Rotherham went straight to the other end in the 15th minute.
A clever pass from Pablo Mills found Alex Rhodes, who left Richard Wood flat-footed and slotted in a cool shot.
Esajas came within a whisker of restoring Wednesday's lead three minutes before half-time.
The home side won a free kick 30 yards out and when James O'Connor - on a season long loan from Chelsea - slipped the ball into his path, Esajas smashed a stunning shot that beat Warrington but hit the woodwork.
Two minutes into the second half Esajas -who switched to the right wing - was the dangerman when he cut inside smartly but Warrington made a great point-blank save.
Ten minutes from the end Wednesday were denied again when a Smith shot hit the post.
Then came the extra-time and penalty drama.