State of GA safety (2005 Nall Report)

I read on AvWeb today the '05 Nall Report is out, and apparently 2004
was a historic low for aviation accidents. This led me to question what
next year's report will look like after considering the spate of GA
accidents that we've all read about in the last year or so. Maybe it's
just a question of perception? Or, are pilots just getting too
complacent when they strap into their aircraft?

I read on AvWeb today the '05 Nall Report is out, and apparently 2004
was a historic low for aviation accidents. This led me to question what
next year's report will look like after considering the spate of GA
accidents that we've all read about in the last year or so. Maybe it's
just a question of perception? Or, are pilots just getting too
complacent when they strap into their aircraft?

Or maybe it's just a statistical artifact. Shift a few accidents from
December to January, and shift a few others from next January to this
December, and you have a banner year for airplane crashes caused simply
by the artificial boundaries of the sample set.

Sometimes random events cluster for no reason. In fact, it is highly
unlikely that they would =not= cluster.

Jose
--
Money: what you need when you run out of brains.
for Email, make the obvious change in the address.

On Thu, 09 Mar 2006 16:16:53 GMT, Jose
wrote:

I read on AvWeb today the '05 Nall Report is out, and apparently 2004
was a historic low for aviation accidents. This led me to question what
next year's report will look like after considering the spate of GA
accidents that we've all read about in the last year or so. Maybe it's
just a question of perception? Or, are pilots just getting too
complacent when they strap into their aircraft?

Or maybe it's just a statistical artifact. Shift a few accidents from
December to January, and shift a few others from next January to this
December, and you have a banner year for airplane crashes caused simply
by the artificial boundaries of the sample set.

Sometimes random events cluster for no reason. In fact, it is highly
unlikely that they would =not= cluster.

Fer shure. Folks should google "Texas Sharpshooter Fallacy."

Don

Or maybe it's just a statistical artifact. Shift a few accidents from
December to January, and shift a few others from next January to this
December, and you have a banner year for airplane crashes caused simply
by the artificial boundaries of the sample set.

Sometimes random events cluster for no reason. In fact, it is highly
unlikely that they would =not= cluster.

Jose

True. From this post and others, I can see that you have a background in
the science of probability, since you are suggesting that the YOY changes
may not be statistically relevant.

I haven't yet read the new Nall report, but I did notice that it is based
upon estimated number of hours flown, per the FAA. No one (here) seems to
question or even want to see the assumptions on the number of hours flown.
Do you know how this number is estimated?

What is curious to me is that when I previously posted the data (from BTS)
about aviation subsidies (by sector)based on hours flown, many people
howled that no one could possibly know how many GA hours are actually
flown. They used that to dismiss the idea that GA is heavily subsidized.
But the Nall study is accepted as science.

I haven't yet read the new Nall report, but I did notice that it is based
upon estimated number of hours flown, per the FAA. No one (here) seems to
question or even want to see the assumptions on the number of hours flown.
Do you know how this number is estimated?

What is curious to me is that when I previously posted the data (from BTS)
about aviation subsidies (by sector)based on hours flown, many people
howled that no one could possibly know how many GA hours are actually
flown. They used that to dismiss the idea that GA is heavily subsidized.
But the Nall study is accepted as science.

I don't know how the number is estimated, but in your two examples the
number is being used in two different ways, and that's an important
difference. In the case of the Nall report (which I have not read), the
item in question is a trend or cluster (or lack thereof). The necessary
key assumption about GA hours is that there be consistancy across the
data sample being used. Systematic error in the number is not anywhere
near as important, since the trends would still show (or not).
Systematic error would tend to cancel out, while random error would not.
(Indeed, random error is what causes the illusion of clusters).

In the case of subsidies, the =actual= number is important. We are not
looking at trends, but rather, whether a certain value is (or is not)
greater than zero. To that end, random error would tend to cancel out,
and systematic error would propagate.

(For those unfamiliar with the terms, systematic error is an error
introduced by a deficiency in the method of measurement (for example,
using a ruler that is too short). It will be the same error every time
you do a measurement. Random error is an error introduced by the slop
in the measurement (for example, using a ruler that changes size with
temperature, and measuring on several different days). This error will
tend to be different for each measurement. It is similar to the
difference between "accuracy" and "precision".)

Jose
--
Money: what you need when you run out of brains.
for Email, make the obvious change in the address.

I don't know how the number is estimated, but in your two examples the
number is being used in two different ways, and that's an important
difference. In the case of the Nall report (which I have not read), the
item in question is a trend or cluster (or lack thereof). The necessary
key assumption about GA hours is that there be consistancy across the
data sample being used. Systematic error in the number is not anywhere
near as important, since the trends would still show (or not).
Systematic error would tend to cancel out, while random error would not.
(Indeed, random error is what causes the illusion of clusters).

In the case of subsidies, the =actual= number is important. We are not
looking at trends, but rather, whether a certain value is (or is not)
greater than zero. To that end, random error would tend to cancel out,
and systematic error would propagate.

True that sampling errors would tend to cancel out in the trend analysis
that Nall uses if the estimating method is statistically valid (truly
random sample, sufficient sample size, etc) and that the BTS number is a
point in time estimate of the subsidy. Meaning that there is standard
deviation around the subsidy per hour flown -- the BTS study might have
provided a range for the subsidy.

But I find it interesting that a key assumption underlying both studies is
questioned in one case, but accepted in the other.

But I find it interesting that a key assumption underlying both studies is
questioned in one case, but accepted in the other.

The key assumption in one case is that the numbers are consistent, in
the other that they are accurate.

True that sampling errors would tend to cancel out in the trend analysis

That's not what I'm saying. I'm saying that the =systematic= errors
would cancel out for the accident study. That is, suppose that the GA
figures are 50% higher than would be accurate. If that 50% figure is
carried throughout the study duration, you will still see (or not see)
the same clusterings of accidents. The actual =number= of accidents per
mile flow will be different, of course, but the existance and location
(in time) of the clusters will be the same.

Were one to attempt to derive a "safety figure" from the numbers, your
objection would be valid. But to just see "this year seemed better by
45% than last year) the systematic error would mostly cancel out (except
in the case of some pathological cases one could come up with).

Systematic and random error are two totally different beasts.

Jose
--
Money: what you need when you run out of brains.
for Email, make the obvious change in the address.

"Skylune" wrote in message
lkaboutaviation.com...
I haven't yet read the new Nall report, but I did notice that it is based
upon estimated number of hours flown, per the FAA. No one (here) seems to
question or even want to see the assumptions on the number of hours flown.
Do you know how this number is estimated?

"GA flight hours are estimated using data from an annual aircraft activity
survey conducted by the FAA. Whether this survey accurately reports the
total hours has been debated for years, but even with likely inaccuracies,
the relationships between accident categories will remain constant." --2005
Nall Report

--Gary

Clustering can reduce the rate for one year and elevate it for another
year. However, the report shows that accident rates have been declining
for the past 10 years, which is a significant point.


Jose wrote:
I read on AvWeb today the '05 Nall Report is out, and apparently 2004
was a historic low for aviation accidents. This led me to question what
next year's report will look like after considering the spate of GA
accidents that we've all read about in the last year or so. Maybe it's
just a question of perception? Or, are pilots just getting too
complacent when they strap into their aircraft?

Or maybe it's just a statistical artifact. Shift a few accidents from
December to January, and shift a few others from next January to this
December, and you have a banner year for airplane crashes caused simply
by the artificial boundaries of the sample set.

Sometimes random events cluster for no reason. In fact, it is highly
unlikely that they would =not= cluster.

Jose
--
Money: what you need when you run out of brains.
for Email, make the obvious change in the address.

Andrew wrote:
Clustering can reduce the rate for one year and elevate it for another
year.

Right, Jose mentioned this in his reply to my OP

However, the report shows that accident rates have been declining
for the past 10 years, which is a significant point

That IS good news. I'm curious to see how/if '05 follows to the overall
trend.