Sources close to Mr Kelly have told Bloomberg that he quit the race after seeing a split between the Bank of America directors who wanted an external candidate and those who wanted to promote someone from inside the company.
He was widely believed to have been the external candidate most likely to land the post.
Bank of America sources have given a different version of events to the Wall Street Journal, claiming that the real reason Mr Kelly withdrew his candidacy was due to a breakdown in talks over how much he would be paid.
He is alleged to have wanted a salary of $20 million, as well as the role of chairman in addition to being CEO.
Mr Kelly is not the first external candidate to turn down the role, with others reported to have been approached in regard to the position including Bob Diamond, president of Barclays, and Citigroup director Michael O'Neill.
With current CEO Kenneth Lewis set to retire at the end of the year, the protracted search for his replacement is continuing.
If a successor is not found in time, it is believed Bank of America may either appoint an interim CEO or ask Mr Lewis to stay in his role until a suitable candidate is found.
Mr Lewis has said he has not been approached about the idea but would consider postponing his retirement if necessary.
Analysts have suggested that Bank of America's chief risk officer Gregory Curl, who has been repeatedly linked with the top job, may have improved his chances of landing the position after helping to negotiate the bank's exit from the Troubled Asset Relief Program earlier this month.
By Claire Archer