Kyle,
Been there, done that, still do it.
About 10 years ago we moved into town because the kids were gone, and our rural area had become part of the Washington DC suburbs. Our taxes had grown equal to our mortgage. We got rid of it all. Our property, my toys, our critters, etc.
Neither my wife, nor I, had ever spent any significant time living in an urban environment, although we worked in the suburbs. It didn't take me long to understand why there is so much urban and suburban violence. I had never before "not met" so many people. We met very few of our new neighbors. If we went for a walk in the evening, we'd pass people who would look away.
Within a year of moving into town, we bought a rural weekend place on seven rather secluded acres. Seven months later, we bought the farmette across the road. We traded our house in town for a townhouse in a high class neighborhood where we stayed during the work week. After drug raids, shootings, battles over parking, being told that our garage door was the wrong color, etc., we'd had enough. We found a great little farm near our weekend place, and it was only an hour commute each way. The townhouse was sold!
Things went well for about a year. Then, the owner of the company where my wife and I worked for quite some time, decided he had made enough money. He basically shut down, keeping just two sorry souls.
For the past three years I've been commuting 75 miles each way, each day. I've just started looking at rentals for me, where I could flop a few nights a week. I just cannot again give up my trees, my beloved domestic and wild critters, my pond, my toys, etc.
Please think long and hard about what you are about to do. I know it is a very difficult decision.
Knute