Even if we completely accept what you are saying, I think there are a couple reasons why that doesn't change anything.
We don't allow people to risk their own lives. We force people by law to wear motorcycle helmets and seatbelts (ignoring a very few number of states). That doesn't at all improve the chances for other people to survive. You aren't considering the fact that a bicyclist causes injuries to people who aren't them. There are plenty of scenarios for this, cars swerving into incoming traffic, into a utility pole into pedestrians to avoid a cyclist. Way more people die from swerving to avoid animals than from hitting animals; even hitting a deer isn't likely to kill you in a modern automobile, but swerving into a tree to avoid one will.
I think you have a valid point -- but my point about asymmetry of risk still stands. Even if cyclists cause more accidents because of this swerving to avoid maneuver, the injury risk to the cyclist in any accident (fatal or otherwise) is still much higher.
We don't allow people to risk their own lives. We force people by law to wear motorcycle helmets and seatbelts (ignoring a very few number of states). That doesn't at all improve the chances for other people to survive. You aren't considering the fact that a bicyclist causes injuries to people who aren't them. There are plenty of scenarios for this, cars swerving into incoming traffic, into a utility pole into pedestrians to avoid a cyclist. Way more people die from swerving to avoid animals than from hitting animals; even hitting a deer isn't likely to kill you in a modern automobile, but swerving into a tree to avoid one will.