Sir Philip Hampton, chairman of the Royal Bank of Scotland, said many people working in the sector are paid "astonishingly high" sums of money.
Speaking to BBC Radio 4, he admitted that the difference between bankers' pay and the pay of other workers is "difficult to defend".
However, Sir Philip warned that if executives in the banking sector are not handed competitive salaries, the very best people could take their skills elsewhere.
"Our top people are very much in demand and we have seen a significant loss of our top people," he commented.
The Royal Bank of Scotland is 84 per cent owned by the taxpayer following its bailout by the government.
Sir Philip's comment came shortly after he said he would listen to concerns among investors about its new system of performance-linked rewards.
Some bosses at the company are able to receive bonuses if the company's share price reaches 50p.
By Asim Shah