Miltonduff Distillery is said to be located on the site of the meal mill of Pluscarden Abbey, six miles Southwest of the town of Elgin and two miles from the abbey itself. The abbey was founded in 1236 by Benedictine monks, destroyed but revived again in the year 1948.

A stone from the original Abbey is retained by the distillery. The distillery’s water source is the Black Burn which flows from springs near the aforementioned Abbey. Thanks to the excellent supplies of water and barley in the ‘Pluscarden’ area of Speyside (where more than 50 illicit stills operated during the 19th century) a lot of distilleries chose this location after the distillation of whisky was legalised in Scotland in the year 1823 through the Excise Act.

Miltonduff distillery was established in 1824 by Robert Bain and Andrew Peary – shortly after the legalisation of whisky production. Rumour has it that it has previously operated as an illegal distillery under the name ‘Milton’. The ‘duff’ addition probably comes from the family Duff that owned the lands surrounding the distillery. In 1866 William Stuart bought Miltonduff distillery. He remained the sole owner for three decades until Thomas Yool & Co. acquired part of the distillery in 1895. At that point they also expanded the distillery – probably due to the ‘whisky boom’ that hit Scotland at the end of the 19th century. Later Thomas Yool & Co. would gain control and ownership of the entire distillery.
In 1936 Miltonduff was acquired by Hiram Walker – Gooderham & Worts Ltd.