William the Conqueror founded it at the end of 1066, after his victory at Hastings, tucking it into the south-east corner of the old Roman town walls. Its core, the White Tower, gives the whole castle its name and was built from 1078. The Tower was added to the UNESCO World Heritage list in 1988 and is looked after by the independent charity Historic Royal Palaces.

Over the centuries the castle has been a royal palace, a store for weapons and treasure, the Royal Mint, a home for exotic animals, an archive and, above all, a prison, which gave English the phrase "sent to the Tower". Its grim reputation owes much to 16th-century propagandists and 19th-century novelists: until the 20th-century world wars only seven people were executed inside the walls, while most executions took place on Tower Hill to the north. Until the reign of Charles II, monarchs traditionally stayed here for a few nights before riding in procession to Westminster Abbey to be crowned.

The layout is made of three wards, or enclosures, and has kept its shape since Edward I finished his rebuilding in 1285. At the centre stands the White Tower, a Norman keep about 27 metres high, built mainly of Kentish ragstone; its original entrance was raised above the ground and reached by a wooden stair that could be taken away under attack. Inside, St John's Chapel keeps the bare, plain look of the Norman period. Around it, the curtain wall of the inner ward is set with 13 towers, among them the Beauchamp Tower, where prisoners of the 16th and 17th centuries carved inscriptions into the walls, and the Bloody Tower, named in the 16th century in the belief that the Princes in the Tower were murdered there. On the river side, St Thomas's Tower became the water-gate later known as Traitors' Gate.

The Crown Jewels have been on public view since 1669 and are since 1994 have been displayed in the Waterloo Block, a former barracks. The display holds 23,578 gemstones, St Edward's Crown, used at the moment of crowning, the Imperial State Crown and the 800-year-old Coronation Spoon, one of the few pieces to survive after Cromwell's Commonwealth had the regalia melted down or broken. Two years after they first went on show, Colonel Thomas Blood tied up the keeper and got his hands on the crown, sceptre and orb, but was caught when the keeper's son arrived unexpectedly. Elsewhere, visitors have been able to walk through the restored rooms of the medieval royal palace since 2006.

The Yeomen Warders, the castle's traditional guard, lead guided tours as well as carrying out ceremonial duties, and together with soldiers from the King's Guard they perform the Ceremony of the Keys each day. One of them, the Ravenmaster, looks after the ravens; at least six are kept at the Tower, following the belief that the kingdom will fall if they leave. From the time of King John until 1835 the castle also housed a royal menagerie of lions, leopards, a polar bear and an elephant, whose animals were later moved to the London Zoo in Regent's Park.