There's also a big distinction between "in town" living and rural living, here. I've done both.
In town, there are legitimate concerns with the construction on one property affecting a neighbor, with everyone jammed onto 1/4 acre or 1/2 acre lots. And once built, even if it violated ordinances, getting it torn down is always a problem. So, permitting prevents such violations from ever getting built in the first place.
Now where it gets ridiculous is in more rural locations. There is no house in my neighborhood on less than 2 acres, and most are closer to 10 acres. Yet, now we have to do all sorts of drainage and runoff studies if the cumulative sum of all our permitted construction (driveways + patios + sheds + buildings) ever exceeds 1000 square feet. On most of these properties, that's less than 0.4% of their land.
But these drainage and runoff studies do keep the engineering office contracted by the township very busy and flush with work. It probably pays the complete salaries of at least a few employees.