At night, it’s harder to see if there is a problem with your tie-downs. Also, if there is a collision, and the boulders get loose, that’s much worse. I’ve seen vehicles totaled and the occupant injured over armadillos. A boulder that gets loose is likely to kill anyone who hits it. At night, things are harder to see, yes. The biggest issue is other drivers will likely be looking at the vehicle that wrecked, not at what may be left in the roadway. I’ve worked a lot of secondary collisions, due to this.
That’s not to say driving at certain times don’t hold benefits. My preference would be a weekend early morning, before dawn, but after “normal” visibility starts.