Castello Brown is a house museum located high above the harbour of Portofino, Italy.
The castle's site is well suited for harbour defence, and appears to have been so used since the 15th century. According to the Record Office of Genoa, cannon batteries were constructed on the site in the early 16th century, and military engineer Giovanni Maria Olgiati drew up plans for a full fortress circa 1554. The resultant castello was completed by 1557, and, in 1575, was instrumental in turning back an attack on the town by Giò Andrea Doria. The structure was enlarged from 1622 to 1624, and survived in this form for a century and a half. The little tower was destroyed in 1798 by an English attack during Napoleon's Ligurian Republic. The castello was abandoned after the Congress of Vienna in 1815.
In 1867, the structure was purchased for 7,000 lire by Montague Yeats-Brown, then English consul in Genoa. He engaged the architect Alfredo D'Andrade, and with advice from his artist friend and fellow-consul James Harris the fort was transformed to a comfortable villa without substantial alteration in its general form. His descendants held the property until 1949, then sold it to an English couple, Colonel and Mrs. John Baber, who restored several ruined sections, until they in turn sold it in 1961 to the City of Portofino.
Elizabeth von Arnim wrote her book The Enchanted April at the castello in 1922. The 1992 movie was filmed there as well.