The clash between the two men is reported to have been caused by the recently botched sale of AIG's Asian unit AIA to Prudential for $35.5 billion - a deal which collapsed after investors in the British company found the price too high.
A reduced takeover was then proffered with the backing of Mr Benmosche, but was rejected by AIG directors, reports Bloomberg.
According to Mr Golub's resignation letter, this led to the pair's relationship becoming "ineffective and unsustainable" in the eyes of Mr Benmosche.
"I'm resigning for the simple reason I believe it's easier to replace a chairman than a CEO," he added.
Director Steve Miller has been appointed as the new chairman.
Last month, a report from the Congressional Oversight Panel suggested AIG may still be able to raise $35.5 billion from AIA, as it is considering the possibility of holding an initial public offering in Hong Kong for the division.
By Asim Shah