Mr Klein, employed by the firm for the past 23 years, is the latest in a series of Citi bosses to quit in the wake of Mr Pandit's appointment late last year.
The bank, one of the worst-affected by the credit crunch, is currently selling off many of its operations in a bid to shore up the balance sheet.
Consumer finance businesses in Germany and Japan have already gone, with more units to follow, as Mr Pandit follows through on his stated aim that the company should "get fit".
Mr Klein said: "It has been particularly important for me to assist the company during this challenging year in the markets and in the management succession at the firm."
Last week, Citi posted a loss of $2.5 billion, less than analysts' expectations.
The bank has written off over $40 billion of assets due to the credit crunch.