It sounds like Gary has hit the nail on the head. I also don't think there is anything wrong with your tractor but you are not running a high enough rpm. I also have one of those tractor's, with the turbo-charged JD Powerstroke engine. It is rather loud. On my open station tractor, running at higher rpm's is a lot more comfortable with ear muffs or plugs. The high rpm engine noise aint horrible though, and I would think it we be quite bearable if you have a cab, without any hearing protection. Lower rpms is quite and more comfortable but does not get you anywhere near the maximum potential hp on your engine's power curve. Pure physics tells us that the steeper the grade, and the higher the speed, the greater the hp requirement. That is what you are seeing first-hand, even with little or no pto or drawbar load.
On my own tractor, most of the implements (disk, plow, rear blade, etc.) I use were designed for a 28 hp, Ford N, and can therefore be operated with the engine at or near idle speed on my pancake-flat farm. It is only when I get into the thicker stuff, with a appropriate-sized, 6 ft, rotary cutter, that I need the rpm's (and the earplugs). Also, with a power-reverser trans, I am not taking the 5-15% hp hit as you are with the hydro. That is not a whole lot, and for you at least, sounds like it is more than made up for by the added convenience.
There are a few reasons why I mostly like to run implements that are on the small side, and one is certainly the ability to get rid of the hearing protection (not especially comfortable). Most importantly for me, is the ability to power right through the mud holes, common on my farm in the spring. without loosing traction. This always keeps my planting from being too late. I especially like watching the wakes roll of the little 2x12 plow as I power through standing water. The only problem here is that you will need a little welding knowledge to "beef-up" them little tools.