Police said at least five civilians had been killed but another Taliban official put the toll as high as 20.
The local offshoot of the Islamic State group, known as Isis-K, claimed it carried out the attack.
It comes after recent blasts targeting foreign interests. Several countries, including Turkey and China, have embassies in the area.
The attack took place about 16:00 local time (11:30 GMT) when the bomber tried, but failed to enter the ministry building itself, the Taliban said.
"I saw the man blowing himself up," said Jamshed Karimi, a driver who was waiting outside the ministry.
Mr Karimi told AFP that he saw a man holding a bag and with a rifle slung over his shoulder walk past. "He passed by my car and after a few seconds there was a loud blast."
The building itself did not appear to be badly damaged. At the nearby interior ministry, window panes were also shattered by the explosion.
But the Italian humanitarian agency, Emergency NGO in Kabul said it had received more than 40 wounded people and the casualty numbers were continuing to rise.
Kabul police described the attack as cowardly, adding in a statement that the perpetrators would be held accountable.
Isis-K said in a post on Telegram that the blast killed at least 20 people. It also said "several 'diplomatic' employees" were among the dead, in a claim that could not be independently verified.
Earlier reports had suggested a Chinese delegation was due to hold talks with the Afghan officials inside the foreign ministry building at the time of the attack.
But a senior official in the prime minister's office told AFP that no foreigners were present.
A string of attacks has targeted foreigners or foreign interests in recent months, at a time when the Taliban is trying to attract investment from neighbouring countries.
Afghanistan has been rocked by dozens of blasts since the Taliban seized power last year, mostly claimed by Isis-K.
Correspondents say that the Taliban tend to underplay casualty figures in such incidents.