The problem you run into with the larger cutter is weight.
Given you said you will be mowing some tough conditions, That puts me thinking you want a heavyduty cutter. And heavy-duty 6' cutters weigh ~1000-1100lbs and if you put one of those on your tractor, you will need front weight. Either suitcase weights OR a front loader that stays ON when you bushhog.
But powerwise, you will be fine. But you really wont save much if any time going to a 6' cutter. If you stay with a 5' cutter which is smaller yes, but you will be able to go faster without bogging down.
In dense and tough stuff where HP limits your speed, larger cutters wont be faster. The only way a larger cutter would be faster on your machine is in light conditions, where the terrain (bumps), limits the ground speed.
I have a L3400 (same tractor just older and 4hp less). I have both a 5' med-duty and a 6' heavy duty. The 5' cutter is just as fast, and is more manuverable because it is smaller, and I can take the loader off and run it.