SEC chairman Christopher Cox said an initial review of the commission's actions regarding the Madoff case had uncovered "deeply troubling" failures by SEC staff to follow-up on warnings of apparent wrongdoing.
Mr Madoff is currently charged with running a $50 billion fraud in one of the biggest cases in Wall Street history.
"I am gravely concerned by the apparent multiple failures over at least a decade to thoroughly investigate these allegations or at any point to seek formal authority to pursue them," Mr Cox said.
He added that the regulator would now undertake a "full and immediate" review of the past allegations against Mr Madoff and his investment company, as well as the reasons they were not judged credible enough to merit further examination.
The review will be led by SEC inspector general H David Kotz.