Yep, you got it. So for getting the pitman arm off get or borrow one of these. Make sure the size is correct for the arms to fit behind the round part.
Pitman Arm Puller & Tie Rod Puller Attached is a pick of the ball joint you want off. Hit with a heavy hammer where the blue arrow points. You can hit where the red arrow points but you would need to hold something heavier than the hammer against the opposite, like a sledge hammer sized item. I use a 4lb hammer, a few whacks heading towards a solid back piece and they pop right off.