Many parts of the rural South can fill that bill. Try the areas around Nashville, Tennessee, and the Smoky Mountain region. The climate's gentle (lower utilities), taxes are super low, in some places electricity is still at super low rates because of the rural electrification act, there's a lot of opportunity for outdoor activities and you can be in a larger city with an hour.
If you like a more snowy climate, the areas outside Lake George NY have a lot of homes that sell inexpensively, and you're only an hour from Albany. The scenery is gorgeous and if you like the outdoor life and four seasons, you can find a lot to enjoy. It's really quiet and low-stress, low crime rate, friendly people and a close-knit community. You have to be able to handle the snow though.