Six families of German descent founded the village in 1811 after the province of Nova Scotia granted them more than 800 acres; they lived mainly from fishing, farmed where the soil allowed and grazed cattle on the land around. By the early 1900s about 300 people lived here, with a school, a church, a general store and a lobster cannery. Artists and photographers came in growing numbers as the roads improved, and after the Second World War tourism outgrew fishing, although lobster fishing has carried on.

In 1766 the cove was recorded as Peggs Harbour or Eastern Point Harbour. The present name probably comes from St. Margarets Bay, since Peggy is a pet form of Margaret; Samuel de Champlain had named the bay after his mother, Marguerite Le Roy. Local legend prefers another story, in which the only survivor of a shipwreck at Halibut Rock grew up here, in 1800 married a local man and came to be called "Peggy of the Cove".

The landscape is bare rock. The granite took shape in the Devonian Period, more than 400 million years ago, as part of the Great Nova Scotia batholith, and glaciers later scraped the ground clean, leaving scoured bedrock and large boulders called glacial erratics strewn across open barrens. To keep it that way, the Peggys Cove Commission Act of 1962 placed some 2,000 acres between Indian Harbour and West Dover, with their barrens, bogs, ponds and rocky coast, under protection from development. In 2021 a viewing deck of 1,300 square metres opened beside the rocks, offering a safer place to watch the sea than the rocks themselves.

William E. deGarthe, a painter and sculptor, made his home in the village, and a gallery shows his work. Beside it, in the William E. deGarthe Provincial Park, he carved a 30 m granite outcrop into a memorial to Nova Scotian fishermen: 32 fishermen with their wives and children under the wings of Saint Elmo, patron saint of sailors, along with the legendary Peggy. Another memory hangs over the bay: on September 2, 1998, Swissair Flight 111 went down about eight kilometres southwest of Peggys Cove with all 229 people aboard, and fishing boats from the cove were among the first to reach the crash site. The small Carpenter Gothic St. John's Anglican Church in the village is a municipally designated heritage site.