Not to pescimistic, but your not going to get rid of them by messing with the hole. They have multiple escape holes or back doors. I've hunted them as a hobby for over 30 years and killed hundreds.
You've got to either killem' or move em'. Here in MD it's against the law to move a captured animal from one location to another. This leaves 1 solution, 22 shorts (CB caps are fine for squirrels but are rather impotent for hogs) out of a bolt action r****.
Semi autos can be hard to feed by hand. Try for a head shot as that will keep them from running off wounded into your neighbors place. 22 shorts are not hypersonic and create no hypersonic crack and a bit less noise. Wait for a moment when no one is around, shoot the hog, put the r**** away, dump him in the trash and then fill in the hole with dirt. Chances are no one will notice the noise.
You can also dump them in a field for vulture chow.
If he gets the drop on you, wait about 20 minutes and he will likely show himself again.
I've always used a .22-250 Ruger and 50 grain bullets, A few years ago I had the ultimate ground hog r**** built to bench rest standards. It fires a 67 grain, 6mm frangible bullet at 3,500 fps, and under 250 yards it blows em' into hyperspace. My personal record is a lasered 432 yards. The r**** is capable of more that I am. It shoots tiny little 1 hole groups.
When I was in my teens my Grand Father had had it with them gettting them getting into his garden. We poured 4 milk jugs (the old fashioned metal kind with a lid) of water into the hole about 10 times with no affect.
If you really want them gone you gotta' kill em'.
Have you considered a second hand crossbow. Very effective at close ranges and no noise.