Other passengers will be on the next flight, and other people will be in
the houses below.

Those other passengers will presumably be people who know the pilot,
and can asess the risk accordingly. The risk to the people in the
houses below from private pilots flying overhead is negligible. In
fact, I don't believe I've ever heard of a private pilot crashing into
a house and killing people. They all seem to have higher ratings.

If you're arguing that this particular safety rule has an overall
negative effect on safety in the long term, I won't argue with you. Of
course it does. All safety rules do. However, this rule does protect
the innocent - those who don't fly, don't have friends who do, and thus
know nothing about GA and can't make an intelligent risk asessment.

Michael