yooperdave,
My parents bought an small older summer cottage in 1973, which had a forced hot air (FHA) propane furnace. We started using the cottage in the winter and by 1975 we replaced the propane furnace with a Sears FHA oil furnace, which significantly lowered our heating cost. This was due solely to the lower $/BTU for oil, and the fact that you own the tank, so you can shop competitively for oil. Also, oil has a lot more energy per gallon, so you can store more energy per fill-up, thus avoiding higher winter fuel prices.
That furnace worked very reliably until my parents sold the cottage in 2000. The only exception was one spring when the cellar flooded all the way up to the oil burner motor and the bearings went bad, had to replace the burner motor, a minor expense.
We did keep a small 100 gallon propane tank for the stove and hot water heater. We spent much more time maintaining the propane hot water heater. The moisture in the dirt floor cellar would rust the cast iron burner and we would have to disassemble it every few years for cleaning, and had to replace the whole unit once.
Others may disagree, but I am not a fan of using propane for heating in cold climates. High efficiency propane heaters are available, but usually have more complexity or non-serviceable parts (i.e. computer boards) to deal with. Oil burners can have the same heating cost with lower efficiency because the fuel costs less, and with very simple controls. Just my $0.02.