The Knights Hospitaller of St John built most of it between their arrival in 1309 and the Ottoman conquest of 1522, which is why its streets are lined with Gothic architecture, later mixed with the mosques, baths and mansions of the Ottoman centuries. UNESCO inscribed it as a World Heritage Site in 1988.

The town has two parts: the high town to the north, built entirely by the Knights, and the lower town to the south-southwest. Its landmark is the Palace of the Grand Master, also called the Kastello, a rare Gothic building on Greek soil. A Byzantine citadel from the late 7th century, it became the Knights' headquarters and the residence of their Grand Master; under the Ottomans it served as a command centre until 1856, when lightning struck a gunpowder store under the nearby Church of St John and the explosion brought down most of the palace's upper floors.

The palace visitors see today is largely an Italian reconstruction. During Italian rule the architect Vittorio Mesturino redesigned the palace and the nearby Street of the Knights, and the palace was rebuilt between 1937 and 1940 with little to go on from before the Ottoman era; scholars have criticised the result for its lack of historical accuracy. It served as a holiday residence for King Victor Emmanuel III and later for Mussolini, whose name survives on a plaque near the entrance. After Rhodes passed to Greece in 1948 the palace became a museum.

The walls show how fortification changed when cannon arrived. The Knights strengthened the Byzantine walls from 1309 onwards, taking the Theodosian walls of Constantinople as their model, and after the Ottoman siege of 1480 and an earthquake they brought in Italian military engineers to rebuild them against artillery. Each stretch was guarded by one of the Knights' national "tongues", and names such as the Bastion of Italy and the terreplein of Spain still carry their names. The Ottomans did not demolish the walls after 1522 but kept them in repair, so the circuit stands much as it was at the end of the Knights' rule.

Eleven gates lead into the old town. The d'Amboise Gate, just below the palace, has a triple defensive belt, two round towers and remains of its old studded wooden doors. The Marine Gate was the main entrance from the harbour. The Gate of St John is known as Porta Kokkini, the "red door", from the tradition that so many men fell before it in 1522 that the stones turned red. Inside, the streets and squares include Socrates Street, Socrates Square, the Jewish Square, the clock tower and the Suleiman Mosque.