16-year-old Harry Cochrane started in midfield for Hearts.
Celtic is just two points clear of second-place Aberdeen in the league standings, with a game in hand.
A 2-1 loss away against St Johnstone in May 2016 was the last time that Brendan Rodgers team had faced defeat anywhere other than European competition until this afternoon.
Leigh Griffiths had the first real sight of goal inside 20 minutes when he lashed a shot into the side-netting from James Forrest's cut-back.
They extended their lead before halftime when Kyle Lafferty profited from a quick break and put the ball past goalkeeper Craig Gordon.
Celtic central defender Dedryck Boyata looks set to make a quick recovery ahead of for Sunday's trip to Edinburgh to take Heart of Midlothian with Tom Rogic the only player that is certainly unavailable for selection.
With 15 minutes to go, Hearts were awarded a penalty when Gordon brought down Ross Callachan and Milinkovic rolled the spot-kick into the bottom corner (76) to consign Celtic to their heaviest domestic defeat in more than seven years.
"We could have played five games today and not scored a goal".
Brendan Rodgers: "I thought they were the better team".
There has been a sense that Celtic have been hanging on to their undefeated run by a thread for several weeks now.
"The players will go down in history because I am not sure it will be done again but certainly in our lifetimes it won't be done and the players can take pride in that".