The executive is to leave his position at the end of the month, to be replaced by the firm's current chief credit officer Andrew Kimball.
Mr Kimon's ousting follows heavy criticism of Moody's, along with other ratings firms, for its apparent over-valuation of complex financial products known as collateralised debt obligations (CDOs).
Several kinds of these derivatives had received the coveted AAA rating from the firm, prior to their value collapsing with the meltdown of the US sub-prime sector.
An inquiry into similar malpractice in the valuation of another type of product, known as constant proportion debt obligations (CPDOs) was also launched earlier this year - and found that a computer error at Moody's led to around $1 billion of the assets being wrongly rated.
Following this report, Moody's chairman and chief executive Raymond McDaniel said: "I am deeply disappointed by the conduct that occurred in this incident. In response, we are taking immediate and appropriate action to address the lapse in our rating process and to ensure that a similar event does not occur again."