Rowe was the site of fishing and foraging for local Native American tribes. The area was first visited by white settlers in 1744, and was the site of a fort to guard against raids. In 1762, the town lands were purchased by the Rev. Cornelius Jones, who named it "Myrifield" after the Greek word for "thousand". The town had enough settlers by 1785 to have it incorporated as a town, renamed by the Massachusetts General Court after John Rowe, a prominent Boston merchant.
The town of Rowe grew around mills on the river, but also had other industries, including sulfur, talc and soapstone mining. This was nowhere more evident than in the now abandoned settlement at the Davis Mine. In the late 1880s, with the addition of the railroad along the river, the area had become somewhat of a small resort town. But, by the advent of the 1900s, most industry had begun to dry up, leaving the town fairly rural until the 1950s. At that point, with the "baby boom" underway, Rowe became the site of Yankee Rowe Nuclear Power Station, the first nuclear power plant in New England, near the Sherman Dam along the Vermont border. The plant was in operation from 1960 to 1992, and the plant is now completely decommissioned, with the nuclear waste planned to be transported to Yucca Mountain's containment facilities, or another facility in the event that Yucca Mountain remains closed as a result of Harry Reid's lobbying to keep the planned nuclear waste repository closed permanently.
Rowe lies along the eastern bank of the Deerfield River, which is dammed near the Vermont border to form the Sherman Reservoir. The town has two other large bodies of water, the Upper Bear Swamp Reservoir and Pelham Lake, which feeds Pelham Brook, a tributary of the river. The town is hilly, with two main ridges on either side of Pelham Brook. Near the southwest corner of town is Negus Mountain, along the western ridge, and along the eastern ridge lies Todd Mountain and Adams Mountain, the highest point in town. Much of the land southeast of Pelham Lake is part of Pelham Lake Park, which extends to the mountains. There are also two small units of Monroe State Forest in town.