Wriggling down could be fixed with another collar below it but I have concerns about the fit between that “u-bolt” and the splined shaft. I worry they you are one bump away from slipping and stripping something.
Maybe the “u-bolt” is as tight as it will get due to wear from slipping up and down?
Personally, I would replace what is out of spec/get better fitment with whatever it takes
Keep us in the loop