Liverpool won their 17th consecutive Premier League game, and their 8th of this season to maintain their 100% Premier League record.
James Milner, back in the starting line up, helped set up Sadio Mane for his 50th Premier League goal before converting an injury time penalty in the 95th minute.
James Maddison had threatened to halt Liverpool's winning run when he equalised 10 minutes from time, but Brendan Rodgers team leave Anfield without any points.
Two changes
Jurgen Klopp had made two changes from Liverpool's narrow win over Salzburg in midweek. Joe Gomez moved to the bench with Dejan Lovren getting a rare start in his place. Jordan Henderson gave up his spot in the starting XI for James Milner.
Brendan Rodgers received warm applause upon his appearance from the tunnel. This his first return to Anfield since he was replaced by Klopp 4 years ago this month.
It was the reds who had the better of the opportunities in the first half - Mo Salah getting an early shot away to test Schmeichel who was defending the Kop goal in the first half.
Alexander-Arnold and Andy Robertson were proving a constant menace - with Trent getting the better of Ben Chilwell on a number of occasions. As the first half approached it's conclusion Leicester sensed they may get something from the game, but that period proved to be their undoing.
James Milner's quick ball transitioned from defence to attack and Sadio Mane had pace to burn before he dispatched the game's opening goal. It was a 50th Premier League goal for the man from Senegal and his 17th goal at Anfield in his last 15 appearances.
He is having a phenomenal 2019.
Second half and the reds couldn't add to their goal tally despite more domination. Leicester collected a couple of yellow cards and Klopp made a double change, Henderson and Origi on for Wijnaldum and Firmino.
Within minutes Leicester had somehow scored an equaliser. Substitute Ayoze Perez had made his impact - it was his pass that found James Maddison who levelled.
Were Liverpool about to not win a league game?
It was looking that way right up until the 93rd minute when Albrighton brought down Sadio Mane. The referee awarded the penalty and then a brief wait for VAR to confirm it and up stepped James Milner.
Despite the delay, the Yorkshireman put the penalty away and the full time whistle followed shortly after.
The reds remain in 1st place with 24 points from 8 games. Onwards!