A mix of Liverpool's youth and squad players overcame a more senior Arsenal team at Anfield after an injury time comeback.
The reds had initially taken an early lead thanks to a Mustafi own goal before going in to the half time break 3-2 down.
Line up
Kelleher, Williams, Gomez, Van den Berg, Milner, Lallana, Keita, Oxlade-Chamberlain, Elliott, Origi, Brewster.
Subs: Adrian, Jones, Kane, Larouci, Chirivella, Clarkson, Koumetio.
Youth get their chance
Jurgen Klopp decided to continue with a policy of playing youngsters. The reds are blessed with midfield backup - so Keita, Oxlade-Chamberlain and Lallana all got rare starts for the reds.
Up front Rhian Brewster started alongside Divock Origi, with Harvey Elliott making his first appearance at Anfield. Elliott became the youngest ever senior Liverpool player to play at Anfield.
Third choice keeper Kelleher started in goal with Neco Williams making his Liverpool debut at right back. Sepp van den Berg made his first start for the club at centre-half, with Joe Gomez also getting the nod. James Milner was used as a makeshift left back.
Great start
Six minutes in and the reds were in front. Ex-Arsenal man Oxlade Chamberlain sending in a cross, and with Rhian Brewster on his shoulder, Mustafi tried to clear but slid the ball past his goalkeeper.
But Liverpool's defence endured their own difficult period midway through the half.
Arsenal were fortunate with their equaliser. Torreira was clearly offside when he latched on to a spilled save by the keeper.
No VAR however for this game.
As it happened: LFC 5-5 Arsenal (5-4 pens)
Martinelli then compounded the reds misery scoring twice within 10 minutes to leave Klopp's boys 3-1 down ten minutes before the break.
Liverpool did rally and a penalty was waved away when Keita's shot hit the arm of Hector Bellerin. But 20 seconds later there was a penalty. Harvey Elliott fouled in the area. Up stepped captain James Milner to cut the deficit. 2-3 at half time.
Second half
Arsenal quickly found themselves with a two goal advantage restored. James Milner with a poor backpass and Maitland-Niles intercepted. Quick thinking by Mesut Ozil to keep the ball in play then enabled Maitland-Niles to enjoy a simple tap in.
Hope was restored when Oxlade Chamberlain fired in a shot before with half an hour left Divock Origi levelled the game at 4-4.
Joe Willock seemed to have won the game with an excellent drive forward and shot before in the fourth minute of five injury time minutes, debutant Nico Williams crossed for Divock Origi to score the final equaliser - making it 5-5.
Penalties
In front of the Kop, Arsenal started the shootout. The first 6 goals all finding the net, before Kelleher dived to his right to save Ceballos' penalty. Origi then scored his penalty to make it 4-3 for the reds.
Maitland-Niles ensured there would be added pressure for the reds fifth penalty taker, 18 year old Curtis Jones, but he fired his penalty in off the post to firmly place Liverpool's name in the Quarter Final cup draw.
The reds may have some serious fixture congestion ahead, but for now the reds and Jurgen Klopp can enjoy a seriously good test for some of Liverpool's fringe players and academy stars.
Another Anfield night for the scrapbook.