Liverpool gifted two first half goals to Arsenal that allowed the Premier League leaders back into this pulsating clash.
The reds, having enjoyed the luxury of a mammoth 5 day break, brought back a few faces who are recovering from injury and arguably should have taken all three points based on the first half display.
Roberto Firmino gave Liverpool the lead twice in the first 20 minutes before at the other end goalkeeping and defensive frailties continue to plague Liverpool. The two soft equalisers gave Arsenal confidence and dented Liverpool's who then had to come form behind in the 90th minute, when substitute Joe Allen fired in substitute Benteke's knock down.
Benteke didn't start the game - Ibe, Lallana and Firmino were chosen to provide the attack against Arsene Wenger's men.
The reds were on the front foot from the first whistle, in the 10th minute Emre Can fired goalbound - his shot parried by Cech and Firmino fired his first of the evening.
Moments later however and Ramsey turned inside the box and fired a low shot that should not have beaten Mignolet at the near post - it did and Arsenal fans were buoyed. Ironic as it may be for the fans from the Emirates they started up the chant of 'Where's your famous atmosphere.' As the last syllable was sung - Firmino smashed in a 20 yard drive and Arsenal fans found the answer to their question.
But Liverpool's second lead of the night wouldn't last either.
Another five minutes later and Arsenal equalised again - this time through a corner that took a weak touch off Giroud and again Mignolet was left looking hapless in the goal as it squeezed inside the post.
Liverpool's determined attacks were being undermined at every attempt.
Klopp cut an animated figure on the touchline. A throw in given the wrong way caused him to react furiously at the fourth official - the long coated Wenger attempting to cool the situation. He offered stares in the direction of Mignolet after both goals - although he kept his opinion professional in the post match interviews.
Back to the second half - and after the events of the first half it was Arsenal who grew in confidence. Their goalscoring machine Giroud struck early in the half and put Arsenal, seemingly, on course for 3 points.
The reds made changes, Benteke up front, new loan signing Caulker sent up front and before that Joe Allen also on the field.
And a last minute equaliser did arrive for the reds - Benteke's height enabling him to reach the cross, head down and it was Joe Allen who was the unlikely hero.
Klopp was delighted but will be left to rue the ongoing mistakes at the back that are derailing any potential progress at every turn.
Next up for Liverpool is Man United at Anfield.
