There are a lot of tractors that have been broken in half at the tranny by counterbalancing too much.
Overloading the hitch will result in damage but if one stays within the reasonable range of ballast it should be fine.
I have heard of damaging front axles due to lack of proper ballast in the rear and rear wheel loosing traction too easily as well. Even the manufacturers recommend ballast on the 3pt hitch.