Hopeless with eleven players on the pitch, Liverpool's fortunes turned when James Milner was sent off after 62 minutes for a second yellow card.
The reds had gone a goal down, from a corner, in the 48th minute but following the sending off the reds improved dramatically and Firmino levelled after reacting to a defensive blunder.
Benteke was sent on for the final 10 minutes and in the 94th minute he won a penalty before dispatching it calmly to complete a remarkable turnaround for the reds.
The reds made a couple of changes to the side that beat Man City earlier in the week. Clyne being given a rare rest - with Moreno replacing him. Sakho too returned to the side to replace Kolo Toure.
Daniel Sturridge was named as a substitute.
In the first half the pace of Palace's attack caused the reds problems. Adebayor in particular causing mischief - creating space for Bolasie to try his luck.
Adebayor saw a header smash back off the crossbar early on in the half - a cross being allowed in because LFC players were more interested in appealing for a referee decision.
The reds failed to muster a chance on target in the first half - such is typical of the reds this season that one decent performance is invariably followed by a below average one.
The reds limped into half time but worse was to follow in the opening 15 minutes of the second half.
Within 3 minutes Palace had a deserved lead - and it came from a corner. Joe Ledley scoring his first goal for a year and Palace on course for their first win of 2016.
On the hour Phil Coutinho came on to replace Flanagan. But before Milner could move to right back he received a second yellow card for a foul near the half way line.
Down to 10 men, the reds rapidly improved as they searched for an equaliser.
And it arrived in the 72nd minute. Firmino picking up the ball after a mistake by Palace keeper Alex McCarthy and placing it in the net to become the reds top scorer this season.
Alberto Moreno saw a thunderous shot returned off the post before in the final minute of injury time Benteke was brought down and the reds were awarded a penalty.
He fired in to give the reds all 3 points and move Liverpool up to 7th.
Next up a double header against Man United.