I've had a 4100 with the factory loader since 2000. My place is only 5 acres and I use the 4100 to mow, move crushed rock, blade snow, box blade, and dig post holes.
I use the 4' Deere bush hog I bought with the tractor as well as a Land Pride 60" finish mower. It's not under powered for my use and in my mind a bigger tractor would be less convenient for mowing.
Keep the 4100 and use it for mowing and other tasks within it's capabilities.