A JPMorgan Chase executive told the Financial Times that the introduction of the levy "will be a factor in the decision" of where the new building is to be located.
It is reported that Jamie Dimon, the bank's chief executive, called UK chancellor Alistair Darling earlier this month to protest about the introduction of the tax.
He told Mr Darling that JPMorgan Chase had been a "good corporate citizen" in London for many years but would be willing to move investment elsewhere if it deemed that the UK was becoming too hostile to the banking sector.
The one-off levy will see banks operating in the country pay 50 per cent tax on any bonus payout of more than Â£25,000 ($40,600) to their staff.
By Claire Archer