Failure mode is one of those "more important factors." It is how suddenly failure occurs after the first crack, hole, or gouge. Nobody talks about it, but it's really, really, really important.
Materials that fail fast are said to fail "catastrophically." Of all materials used in bikes, none fails more catastrophically than carbon fiber, and none fails more slowly than steel. You want your bike stuff to respond to trauma by bending and denting, not shattering and snapping. Metals tend to do that. And once that's covered, you want plenty of time and lots of warning between the onset of failure (a crack, for instance) and total material separation. Steel is the first place winner here, too. Reparability is desirable, too, and steel wins that one, also.
Another quality to consider in a frame material is how well it ages; the degree to which it stays strong as it gets old, and environmental stress in the form of ozone, ultraviolet radiation, salt air, and temperature extremes affect it.
In this regard, metals are far superior to rubber, plastic, and carbon fiber. The resins used to hold the layers of carbon fiber together degrade with exposure to ultraviolet.
Defect tolerance is the ability of a material to be safe even when defective.
It matters because perfect quality control is impossible, no matter how white-coated engineers in sterile rooms you have monitoring production. Some bug or booger or bubble will work its way in, and then what?
Then you want a material that maintains its integrity.
The least defect-tolerant material used in bikes is carbon fiber. (And most carbon fiber comes from China, as a matter of fact.) The most defect-tolerant is steel.