Is the tractor new to you or have you had it a while and know how the 4x4 shifter is supposed to work and feel? One of my tractors is pretty much the same and there's an obvious in and out position for that shifter.
I'd suggest the first thing would probably be for someone to get underneath and look at the linkage to see if it's bent, broken, has come apart or loose somewhere, or is bound up with accumulated crud. Those should be pretty obvious upon inspection and probably the easiest to fix. Maybe a helper can operate the 4x4 lever while you or whoever is observing underneath, on a flat, level surface with the engine off and brakes locked, of course. If it all looks OK and seems to operate normally external to the transmission, it indicates something is likely wrong internally.
By the way, welcome to TBN.