A few states, like Colorado, have boundaries based entirely on the graticule of latitude and longitude (the rectangular grid that defines places in relation to Greenwich England and the Equator) rather natural features:
Take a look at the state and county boundaries of Virginia, however:
In Virginia, natural features (especially rivers and divides) as well as straight survey lines were used to define Virginia's state and county boundaries. Notice how two natural features form county boundaries along their entire length - the Blue Ridge (in blue) and the Rappahannock/Rapidan rivers (in white):
Most states, especially those created initially as English colonies, used natural features as the starting point even for the straight lines you see today on the map. In colonial charters, Maryland's western boundary was defined as a line extending north from the headspring of the Potomac River, while Pennsylvania's western boundary was defined as 5 degrees west of its eastern boundary. Not surprisingly, both colonies sought to define the starting point for those lines in a way that would maximize their territory... and reduce the size of Virginia.
Obviously the Atlantic Ocean marks the eastern boundary of Virginia, as Thomas Jefferson recorded in Notes on the State of Virginia. A non-natural, almost-straight line based on latitude marks the southern boundary with North Carolina. With a simple glance, it appears the Potomac River marks the northern boundary... though in fact there's more to that "edge" than meets the eye. And the western boundary, stretching from Loudoun to Lee counties - that's not a river, but is it based exclusively on watershed divides?