I did that with mine for a couple of years before I filled it with concrete, just used 3/4 crushed stone, the pipe shifted a little but no big deal. the stone was not quite enough weight though. Concrete seems to be just right, I've never wished it was lighter or heavier since I put it in.
The more I think about it, steel pipe welded to the outside might even be better, could always remove them or replace them if they get damaged. leaves the whole inside available for
ballast or empty.
For me I always need the ballast, when ever I try to use the tractor without the ballast it wants to lay down on it's side on me
JB.