The cardinal sketched the building himself and Flaminio Ponzio worked his ideas into a villa suburbana, a country house on the edge of the city, with Giovanni Vasanzio also credited with its design. The gardens around it, once part of the same estate, are now treated as a separate attraction.

When the villa was built, the land lay outside the walls of Rome, reached most easily from the Porta del Popolo, and the grounds had a circumference of nearly 5 km. Around 1775 Prince Marcantonio IV Borghese had the architect Antonio Asprucci redecorate the interior and rearrange the sculptures by theme, and by the end of that century the villa was working as a public museum. The family lost part of its treasure in 1808, when Prince Camillo Borghese, brother-in-law of Napoleon, was forced to part with the ancient marbles, which Napoleon acquired; the Borghese Gladiator and the Borghese Hermaphroditus went to the Louvre. In 1902 the Italian state bought the villa together with the whole estate and its parkland.

The collection is laid out in twenty rooms over two floors. Several of Bernini's sculptures stand in the very rooms they were made for, which gives the gallery a large part of his secular work: the early Goat Amalthea with Infant Jupiter and Faun of 1615, Aeneas, Anchises and Ascanius, the Rape of Proserpine of 1621–1622, Apollo and Daphne of 1622–1625 and the David of 1623. The gallery holds both of Bernini's portrait busts of Scipione Borghese; he made the second after a large crack appeared in the marble of the first. In Hall I, Antonio Canova's Pauline Borghese as Venus Victrix gives the room its name.

The main hall, the Salone, is crowned by a ceiling fresco by Mariano Rossi of Marcus Furius Camillus lifting the Gauls' siege of the Capitoline Hill, painted with such foreshortening that it seems almost three-dimensional. The entrance hall displays a Gladiator Mosaic of 320–330 AD found on the Borghese estate at Torrenova in 1834, next to a Roman statue of Augustus. In the hall of the Hermaphroditus, a sleeping figure lies on a marble mattress that Bernini carved when he restored it.

Scipione backed Bernini from the start of his career and was a keen collector of Caravaggio, so the painter's Boy with a Basket of Fruit, Sick Bacchus and St Jerome Writing are all here. Titian is represented by Sacred and Profane Love and Raphael by the Entombment of Christ and the Young Woman with Unicorn. Further rooms hold tondi by Botticelli and Fra Bartolomeo, The Last Supper by Bassano, Correggio's Danaë and Rubens's Susanna and the Elders. Other museums close by include the Galleria Nazionale d'Arte Moderna, devoted to Italian art of the 19th and 20th centuries, and the Museo Nazionale Etrusco.