You should read zero ohms....if the points in the relay are good, there is no resistance.
And unless the relay has only 3 terminals (one 12v +, one ground for the holding coil, one output to the glow plugs), the typical 4 terminal relay will not tap the holding coil circuit, so, yes, there will be no voltage. One purpose of a relay is to use a light gauge wire for the coil circuit, and a heavier wire to send current on out to whatever is on the working end of that circuit.
Use your meter to check continuity if you can. Engage the holding coil of the relay, check for continuity across the input/output terminals. Drop the holding coil out (take power off), continuity should stop.
Or connect 12v to one terminal of the input/output, engage the holding coil, put a test light on the other terminal to ground and see if it lights. If it does, relay is working. (could STILL have pitted contacts on the connection points, but if you read zero ohms, less chance of that....bad contacts generally show as some resistance to flow)
Assuming you find the relay good, next thing you'll need to check is for a timer. Many glow plug systems today use a timer in series with the relay to let power flow to the relay/glow plugs for a short amount of time, then shut the power off to them.
IF you have a glow plug light on the dash that comes on when you turn the key to the ON position, then it goes out after a period of 30 seconds to a minute, you have a timer. A bad timer could fail in the open position, never allowing current to flow to the relay.
A wiring diagram would be real handy in troubleshooting this situation.