Local residents and witnesses said on Wednesday that the forces from the UN-backed Government of National Accord (GNA) overran the headquarters belonging to the militia, whose members are mainly from Misrata, the hometown of the former prime minister, Khalifa Ghweil.
"It's over. Ghweil's forces have pulled out and GNA forces have taken control of the area," media outlets quoted a witness as saying.
A security source also confirmed the militia's withdrawal.
The unity government forces launched an assault on the headquarters in Guest Palace, a complex of luxury villas in the city center, three days ago. They overran it after heavy fighting and artillery exchanges overnight.
Ghweil, whose administration was replaced by the GNA last year, has stepped up a campaign against the UN-backed authority.
A medic said a rocket hit Khadhra Hospital without causing any casualties.
Gunmen also stormed the headquarters of the privately owned Nabaa television, known for its Islamic leanings. The channel remained off the air on Wednesday.
The ongoing fierce fighting has brought life in the capital to a standstill with schools and shops closed.
The fresh wave of violence comes despite an appeal by Martin Kobler, the UN's Libya envoy, on Tuesday for an "immediate ceasefire."
"Civilians at grave risk in ongoing clashes," Kobler said on Twitter.
A series of violent clashes also erupted in the neighborhoods of Hay al-Andalus and Gargaresh on Monday evening. The fighting prompted the unity government to deploy tanks in the violence-hit areas.
On Tuesday, heavy fighting rocked eastern Libya, where forces loyal to General Khalifa Haftar regained control of two major oil ports of Ras Lanuf and al-Sidra from a rival faction that seized them earlier this month.
Haftar's forces do not recognize the UN-backed government. They mounted a day-long assault by land, sea and air to seize the oil terminals.
Haftar was an ally of Libya’s former long-time dictator Muammar Gaddafi but joined the Libyan revolution against Gaddafi in 2011.
Libya has been rocked by violence since a NATO military intervention that followed the 2011 uprising and led to the overthrow and death of Gaddafi. Rival governments were set up in Tripoli and eastern Libya in 2014.