The stadium shook in the third minute of stoppage time as Colombia defender Yerry Mina rose above the crowd and directed a powerful header off the ground and past Pickford.
Following a nail-biting and bad-tempered affair in Moscow, the game was decided by penalty kicks, and it was England who held their nerve and booked their place in the quarter finals after winning the shootout 4-3.
Earlier things had looked set to go pear-shaped when Jordan Henderson's penalty was saved, only for Mateus Uribe's spot-kick to rattle the bar and goalkeeper Jordan Pickford to deny Carlos Bacca.
Now with a quarter final against Sweden in Samara to look forward to on Saturday, the England boss will make sure he eradicates any complacency ahead of the game against a side that the Three Lions are yet to beat in a World Cup encounter.
Jose Pekerman would have been hoping that the 26 year old could repeat the trick four years later in Russian Federation but instead he had to start the loss against Japan without his talisman and against England Rodriguez didn't even make the bench.
"It was a good match, but (I'm) always disappointed when the game goes to penalties - it doesn't show the true potential of the team", said Emma Kinsey, 32, a yoga instructor from Lincoln who lives in Bogota.
Beer sprayed high into the blue Tyneside sky as Kane scored England's spot-kick.
Keiran Trippier, who scored England's critical fourth penalty to level the shoot-out at 3-3, also tweeted: "What a special night!"
England's hero Eric Dier.
England were the better side without looking too unsafe but were then gifted the lead when Carlos Sanchez hauled down England's captain and talisman Kane once too often.
Mina's leveller put the celebrations on hold, but Pickford, who saved two penalties at the Federation Internationale de Football Association U17 World Cup in 2011, had the last laugh in the shoot-out.
"I've got power and agility, I don't care if I'm not the biggest keeper".
But despite appearing not to see the clash, Geiger issued a yellow card, prompting derision from some former referees - Englishmen Keith Hackett and Mark Clattenburg among them - on social media, questioning when a deliberate head-butt became a yellow card offence. I apologise to all the Colombian people, but the players are not to blame for this.
Rodriguez arrived at the World Cup on the back of a strong season at Bayern Munich, having joined the German giants on a two-year loan from Real Madrid in the summer of 2017. Their service includes Fox Sports 1, Fox and Telemundo (in most markets) so you can stream the game with English or Spanish commentary. "Colombia died standing. That's why I applaud them".
If England keeps advancing, we're all bound to come familiar with it.