This magnificent building that now houses The Captain Jefferds Inn, was originally built in 1804 by Daniel Walker. Upon completion in 1805, it was given as a wedding present to his daughter Mary and her new husband Captain William Jefferds. Mr. Jefferds was a seafaring captain and it is said that his wife Mary would stand on top of the still present widows walk, to watch for her husbands ship to return from sea. The mansion stayed in the Jefferds family until 1884 when it was purchased by the Agnew family. The second and third floors of the house were remodeled by the Agnews in 1888. In 1944 the home was purchased by the Doughterty family. The Doughertys converted what is the present day carriage house suites, to apartments which were rented seasonally.