Mr Khan, who would be the city’s first Muslim mayor, is on course for victory over Conservative Zac Goldsmith.
The result would bolster leader Jeremy Corbyn after Labour were beaten into third in Scotland by the Tories and lost English councillors.
In Scotland, the SNP said it would form a minority government after winning its third election in a row in Scotland.
First Minister Nicola Sturgeon is celebrating what she said was an “emphatic” victory, her first as party leader, after the SNP emerged as the largest party at Holyrood with 63 seats, ahead of the Conservatives on 31 and Labour on 24.
But she played down talk of another independence referendum after falling short by two seats of an overall majority.
In Wales, Labour remains as the largest party, with 29 out of 60 seats, but was denied a majority as Plaid Cymru and UKIP both made notable gains. Counting is continuing in Northern Ireland although it has been a good day so far for the DUP and Sinn Fein.
Mr Khan’s expected victory would end nearly a decade of Conservative control of City Hall. The former Labour MP and minister, 45, would become London’s third mayor after Mr Johnson and Ken Livingstone.
Although the result has yet to be declared he got 44.2% of first preference votes to Mr Goldsmith’s 35.6% – second preference votes are now being counted with Mr Khan set to pass the crucial 50% mark when they are added in.
Mr Khan distanced himself from Mr Corbyn during the campaign, pledging to freeze fares on the capital’s transport network and build more affordable housing, but also promising to champion business and cut taxes on enterprise.
His victory follows a controversial campaign in which the Conservatives were accused of trying to smear Mr Khan by accusing him of sharing a platform with extremists – tactics defended by ministers but questioned by some in the party.