It began as a small mud fort of the Kakatiya dynasty; the name is thought to come from the Telugu Gollakonda, "shepherd's hill". Sultan Quli, sent by the Bahmani kings as governor, declared independence in 1518 and ruled the new Sultanate of Golconda from it, and over 62 years the first three Qutb Shahi sultans rebuilt the mud walls in granite. The capital moved to Hyderabad in 1590.

Golconda's name spread far beyond India because of the diamond mines nearby, above all the Kollur Mine, which made the city a market for large stones known as Golconda diamonds; the fort is said to have had a vault where the Koh-i-Noor and the Hope diamond were once kept. In the early 17th century it was also a centre of cotton weaving, exporting muslin and calico to Persia and European countries and printed cloth to Java and Sumatra. In 1687, after an eight-month siege, the Mughal emperor Aurangzeb took the fort and captured the last Qutb Shahi king, Abul Hasan Qutb Shah, and the citadel fell into ruin.

The outermost enclosure is entered through the Fateh Darwaza, the Victory Gate, named after Aurangzeb's army marched in through it, and studded with giant iron spikes to stop elephants battering it down. Under the dome at the entrance, a hand clap at a certain spot carries all the way up to the Bala Hisar pavilion at the summit, almost a kilometre away, a device that once served as an alarm. The Bala Hisar gate, the main entrance on the east side, has a pointed arch carved with scrollwork, mythical lion-like yalis and peacocks with ornate tails.

Inside, the ruins include royal apartments and halls, the Taramati mosque, the mosque of Ibrahim, armouries, a camel stable, a mortuary bath, the Habshi Kamans arches and the Durbar hall at the top of the hill, with a baradari pavilion on the summit looking out over Hyderabad. The Jagdamba temple next to the king's palace and the mosque of Ibrahim dates back to the early Kakatiya period, and draws hundreds of thousands of Hindu devotees during the Bonalu festival; a Mahakali temple stands nearby. About a kilometre north of the outer wall lie the Qutb Shahi tombs, set among gardens. Golconda, the Charminar and the Qutb Shahi tombs are on India's tentative list for UNESCO World Heritage status.