Old Hydrotherm problem

recently i was working on an old hydrotheerm boiler and the original problem was that it was leaking gas.  It was a  hot water, nat. gas fired boiler with a power pile generator.  After turning the gas on to the house i saw that that pilot cock was still on (with the pilot out) and still pouring gas into the boiler even though the pilot was out.  I wasnt happy with this setup at all because the tapping for the pilot cock was before the main shutoff on the boiler, also there was no controlling the pilot (besides the pilot cock) so if it blew out for any reason (like it did) you would have the same problem of leaking gas.  I changed out the powerpile and re lit the pilot.  My question is that is this an unsafe or unpractical setup, because i think it is.