Newport had led through Jamille Matt's early header but Leicester's pressure finally told eight minutes from time when Rachid Ghezzal levelled with a powerful strike from the edge of the penalty area.
Phil Foden, the England Under-21 global, got on the end of a cross from Ilkay Gundogan in the 43rd minute to score his first home goal for City before Rotherham defender Semi Ajayi turned the ball into his own net.
Remarkably, both Oldham and Barnet have caretaker managers - Pete Wild and Darren Currie - who were taking charge of their respective teams for only the third time.
At Craven Cottage, Denis Odoi had put Claudio Ranieri's side ahead in the 52nd minute, calmly finishing after a penalty box scrap.
Aleksandar Mitrovic could not repeat Surridge's feat when his spot-kick was parried wide by Daniel Iversen, with the on-loan Leicester goalkeeper's heroics paying further dividends in the 88th minute.
United were the victims of a massive shock - Shaquile Coulthirst's 21st-minute penalty securing a famous win for Barnet, the lowest-ranked side now left in the competition.
"I am really proud of them", Currie, who made more than 100 appearances for the club just north of London, said.
"I thought we played well, especially the second half". We had loads of half-chances, some very good positions where we should score, and on another day they all go in.
Bidwell then headed in the victor with 15 minutes to go from a set-piece.
On Saturday, FA Cup holders Chelsea cruised to a 2-0 win over Nottingham Forest in what seems certain to have been Cesc Fabregas´s final match for the Blues.
The other Premier League side to be upset was struggling Cardiff City, who fell 1-0 at Gillingham after naming a heavily-rotated side for the clash at Priestfield Stadium.
Elsewhere, League One Doncaster enjoyed a superb 3-1 away win at Championship side Preston.