The Greek Revival Rotch-Jones-Duff House was built in 1834 for whaling merchant William Rotch, Jr. The interior is furnished to the different periods of the residence's occupation (1834-1981). The home was also occupied in turn by the Jones and Duff families, ship's agents and coal, whale oil, and oil transportation professionals. The surrounding gardens most closely reflect the period 1851 to 1935.

The museum offers period rooms, exhibits, 30-minute self-guided tours of the home, gardens, one-hour guided group tours, guided group tours with tea or a luncheon, fourth and fifth grade educational programs, lectures, and educational programs. The website offers background information for the educational programs.