State of GA safety (2005 Nall Report)

[[email protected]]

I read on AvWeb today the '05 Nall Report is out, and apparently 2004
was a historic low for aviation accidents. This led me to question what
next year's report will look like after considering the spate of GA
accidents that we've all read about in the last year or so. Maybe it's
just a question of perception? Or, are pilots just getting too
complacent when they strap into their aircraft?

[Jose]

> I read on AvWeb today the '05 Nall Report is out, and apparently 2004
> was a historic low for aviation accidents. This led me to question what
> next year's report will look like after considering the spate of GA
> accidents that we've all read about in the last year or so. Maybe it's
> just a question of perception? Or, are pilots just getting too
> complacent when they strap into their aircraft?

Or maybe it's just a statistical artifact. Shift a few accidents from
December to January, and shift a few others from next January to this
December, and you have a banner year for airplane crashes caused simply
by the artificial boundaries of the sample set.

Sometimes random events cluster for no reason. In fact, it is highly
unlikely that they would =not= cluster.

Jose
--
Money: what you need when you run out of brains.
for Email, make the obvious change in the address.

[Don Tuite]

On Thu, 09 Mar 2006 16:16:53 GMT, Jose >
wrote:

>> I read on AvWeb today the '05 Nall Report is out, and apparently 2004
>> was a historic low for aviation accidents. This led me to question what
>> next year's report will look like after considering the spate of GA
>> accidents that we've all read about in the last year or so. Maybe it's
>> just a question of perception? Or, are pilots just getting too
>> complacent when they strap into their aircraft?
>
>Or maybe it's just a statistical artifact. Shift a few accidents from
>December to January, and shift a few others from next January to this
>December, and you have a banner year for airplane crashes caused simply
>by the artificial boundaries of the sample set.
>
>Sometimes random events cluster for no reason. In fact, it is highly
>unlikely that they would =not= cluster.
>
Fer shure. Folks should google "Texas Sharpshooter Fallacy."

Don

[Skylune]

>>Or maybe it's just a statistical artifact. Shift a few accidents from
December to January, and shift a few others from next January to this
December, and you have a banner year for airplane crashes caused simply
by the artificial boundaries of the sample set.

Sometimes random events cluster for no reason. In fact, it is highly
unlikely that they would =not= cluster.

Jose<<

True. From this post and others, I can see that you have a background in
the science of probability, since you are suggesting that the YOY changes
may not be statistically relevant.

I haven't yet read the new Nall report, but I did notice that it is based
upon estimated number of hours flown, per the FAA. No one (here) seems to
question or even want to see the assumptions on the number of hours flown.
Do you know how this number is estimated?

What is curious to me is that when I previously posted the data (from BTS)
about aviation subsidies (by sector)based on hours flown, many people
howled that no one could possibly know how many GA hours are actually
flown. They used that to dismiss the idea that GA is heavily subsidized.
But the Nall study is accepted as science.

[Jose]

> I haven't yet read the new Nall report, but I did notice that it is based
> upon estimated number of hours flown, per the FAA. No one (here) seems to
> question or even want to see the assumptions on the number of hours flown.
> Do you know how this number is estimated?
>
> What is curious to me is that when I previously posted the data (from BTS)
> about aviation subsidies (by sector)based on hours flown, many people
> howled that no one could possibly know how many GA hours are actually
> flown. They used that to dismiss the idea that GA is heavily subsidized.
> But the Nall study is accepted as science.

I don't know how the number is estimated, but in your two examples the
number is being used in two different ways, and that's an important
difference. In the case of the Nall report (which I have not read), the
item in question is a trend or cluster (or lack thereof). The necessary
key assumption about GA hours is that there be consistancy across the
data sample being used. Systematic error in the number is not anywhere
near as important, since the trends would still show (or not).
Systematic error would tend to cancel out, while random error would not.
(Indeed, random error is what causes the illusion of clusters).

In the case of subsidies, the =actual= number is important. We are not
looking at trends, but rather, whether a certain value is (or is not)
greater than zero. To that end, random error would tend to cancel out,
and systematic error would propagate.

(For those unfamiliar with the terms, systematic error is an error
introduced by a deficiency in the method of measurement (for example,
using a ruler that is too short). It will be the same error every time
you do a measurement. Random error is an error introduced by the slop
in the measurement (for example, using a ruler that changes size with
temperature, and measuring on several different days). This error will
tend to be different for each measurement. It is similar to the
difference between "accuracy" and "precision".)

Jose
--
Money: what you need when you run out of brains.
for Email, make the obvious change in the address.

[Gary Drescher]

"Skylune" > wrote in message
lkaboutaviation.com...
> I haven't yet read the new Nall report, but I did notice that it is based
> upon estimated number of hours flown, per the FAA. No one (here) seems to
> question or even want to see the assumptions on the number of hours flown.
> Do you know how this number is estimated?

"GA flight hours are estimated using data from an annual aircraft activity
survey conducted by the FAA. Whether this survey accurately reports the
total hours has been debated for years, but even with likely inaccuracies,
the relationships between accident categories will remain constant." --2005
Nall Report

--Gary

[Skylune]

I don't know how the number is estimated, but in your two examples the
number is being used in two different ways, and that's an important
difference. In the case of the Nall report (which I have not read), the
item in question is a trend or cluster (or lack thereof). The necessary
key assumption about GA hours is that there be consistancy across the
data sample being used. Systematic error in the number is not anywhere
near as important, since the trends would still show (or not).
Systematic error would tend to cancel out, while random error would not.
(Indeed, random error is what causes the illusion of clusters).

In the case of subsidies, the =actual= number is important. We are not
looking at trends, but rather, whether a certain value is (or is not)
greater than zero. To that end, random error would tend to cancel out,
and systematic error would propagate.<<

True that sampling errors would tend to cancel out in the trend analysis
that Nall uses if the estimating method is statistically valid (truly
random sample, sufficient sample size, etc) and that the BTS number is a
point in time estimate of the subsidy. Meaning that there is standard
deviation around the subsidy per hour flown -- the BTS study might have
provided a range for the subsidy.

But I find it interesting that a key assumption underlying both studies is
questioned in one case, but accepted in the other.

[Skylune]

Kingfish said:
Ya know, I kinda figured this topic would flush you out of your hiding
place... :) And, true to form, you've managed to work in a
reference to GA subsidies. Hell, my bowels should be so regular...

OT: Skylune, what did you think about the drunk getting busted on the
ramp at Chicago Midway on Tuesday? That's NEVER happened at my airport
(BDR). I guess the FAA could learn a thing or two about airport
security from GA fields?<snicker>
<<


C'mon King, what hiding place? ;-). There hasn't been much news that I'm
interested in lately, and I don't like to engage in ****ing matches just
for the hell of it.....

On Chicago: surely you jest! Only THEFTS of GA planes have made the
news. Would an unauthorized partly merely being near the runway at a GA
airport even be reported?? I'd say not, but I'll bet it happens.

When planes are STOLEN from large commercial airports (not hijacked, for
those ready to make a 9/11 analogy) at a greater rate than GA airports,
then I think you can snicker for real. ;-)


(Sorry 'bout not having switched to Google group yet. )

[Denny]

Hmmm, it would seem that wholesale deliveries of 100LL would be a
constant that one could use to estimate the annual number of GA flying
hours... The industry reports those sales with great regularity
because the 100LL refiners use that data to plan production...
And, insurance companies get an annual report of claimed hours flown
for setting the premium for the coming year, another possible yardstick
that is industry published...

Now as to those fabulous subsidies to GA... It seems that I am not
getting any of that sugar spread on me and I want to know where I can
file a claim so I can be reimbursed for the 60+ years I have been
banging around in airplanes and not even one damn penny has ever shown
up in the mailbox... I gotta tell ya that I'm really torqued off over
being short sheeted all these years..

denny

[Jose]

> But I find it interesting that a key assumption underlying both studies is
> questioned in one case, but accepted in the other.

The key assumption in one case is that the numbers are consistent, in
the other that they are accurate.

> True that sampling errors would tend to cancel out in the trend analysis

That's not what I'm saying. I'm saying that the =systematic= errors
would cancel out for the accident study. That is, suppose that the GA
figures are 50% higher than would be accurate. If that 50% figure is
carried throughout the study duration, you will still see (or not see)
the same clusterings of accidents. The actual =number= of accidents per
mile flow will be different, of course, but the existance and location
(in time) of the clusters will be the same.

Were one to attempt to derive a "safety figure" from the numbers, your
objection would be valid. But to just see "this year seemed better by
45% than last year) the systematic error would mostly cancel out (except
in the case of some pathological cases one could come up with).

Systematic and random error are two totally different beasts.

Jose
--
Money: what you need when you run out of brains.
for Email, make the obvious change in the address.

[Skylune]

by Jose > Mar 9, 2006 at 08:06 PM

The key assumption in one case is that the numbers are consistent, in
the other that they are accurate.


Systematic and random error are two totally different beasts.

Jose

<<

I do understand what you are saying: I learned the important distinction
between "precision" and "accuracy" many years ago as an undergrad science
student.

The BTS conclusion depends upon the accuracy of the "hours flown"
estimate. The Nall Study's validity depends more upon the precision
(using a method that can repeatedly reproduce the same result within a
small margin of error) of the estimating technique than on its absolute
accuracy. Thus even if the hours flown estimate is "systematically"
misestimated (i.e. is inaccurate) it would not invalidate the conclusions
of the trend analysis conclusion. Yes?

But this is really splitting hairs. The BTS study showed subsidies so
large that even if there was huge sampling error or other statistical
mis-steps that resulted in a vast underestimation of hours flown (and
therefore exaggerated GA subsides), the conclusion is still valid: that
GA is very heavily subsidized relative to other modes of transportation.


Just looking at the raw revenue number from the funding sources also makes
this quite obvious. (I'd repost the BTS data, but I've been yelled at
already for reposting the stuff....)

[Jose]

> Thus even if the hours flown estimate is "systematically"
> misestimated (i.e. is inaccurate) it would not invalidate the conclusions
> of the trend analysis conclusion. Yes?

Yes.

> The BTS study showed subsidies so
> large that...

I haven't read the study so I can't really comment on it intelligently.
But like the difference between precision and accuracy, there is a
difference between pro-rated costs and marginal costs which needs to be
taken into account. One must also consider who benefits - it's not just
the immediate users.

And even if xxx were subsidized, that's not necessarily a bad thing.

Jose
--
Money: what you need when you run out of brains.
for Email, make the obvious change in the address.

[Capt. Geoffrey Thorpe]

> wrote in message
ups.com...
>I read on AvWeb today the '05 Nall Report is out, and apparently 2004
> was a historic low for aviation accidents. This led me to question what
> next year's report will look like after considering the spate of GA
> accidents that we've all read about in the last year or so. Maybe it's
> just a question of perception? Or, are pilots just getting too
> complacent when they strap into their aircraft?
>

You mean "considering the spate of people posting every GA accident they
come across"...

I don't see how the morbid fascination of some individuals here could have
any effect on the actual number of accidents.

--
Geoff
The Sea Hawk at Wow Way d0t Com
remove spaces and make the obvious substitutions to reply by mail
Spell checking is left as an excercise for the reader.

[[email protected]]

>>>>You mean "considering the spate of people posting every GA accident they
come across"...<<<<

Well, as I am a frequent visitor to this forum I suppose that could
have influenced my perception.

>>>>I don't see how the morbid fascination of some individuals here could have
any effect on the actual number of accidents. <<<<

It doesn't.

[[email protected]]

>>>>C'mon King, what hiding place?<<<<

Uh, your cave?

>>>>I don't like to engage in ****ing matches just for the hell of it..<<<<

Really? This can't be the same Skylune we all know and love here on
RAP...

>>>>On Chicago: surely you jest!<<<<

I'm not jesting. And don't call me shirley. (you asked for that)

[almostthere]

For those who want to do their own statistical analysis, this is an
interesting site:

http://www.ntsb.gov/ntsb/query.asp


Limit the date range to Jan. 1 - Dec. 31, 2005 and plug in your favorite
manufacturer in the Make/Model box.
You'd be amazed.




> wrote in message
ups.com...
>I read on AvWeb today the '05 Nall Report is out, and apparently 2004
> was a historic low for aviation accidents. This led me to question what
> next year's report will look like after considering the spate of GA
> accidents that we've all read about in the last year or so. Maybe it's
> just a question of perception? Or, are pilots just getting too
> complacent when they strap into their aircraft?
>

[Dylan Smith]

On 2006-03-09, > wrote:
> I guess that's the real *big picture* view then. I'm not immune to
> altered perceptions when it seemed for a while there were 1 or 2 GA
> accidents a week.

One or two a week? There are almost always at least a dozen GA accidents
a week. If you go to the 'browse' view on the NTSB website, you'll see
there are far more days with multiple GA accidents than there are days
with no accidents at all. I went to September 2005 (picked at random, no
reason) and there were NO days in September 2005 where there were not at
least 2 GA accidents. Most days had about 5 or 6 GA accidents. There
were 31 fatal GA accidents in September 2005 - one more than September
has days.

You're probably only percieving a cluster because they have been talked
about more, but really - the accidents that have been talked about here
recently are not out of the ordinary (sadly).

--
Dylan Smith, Port St Mary, Isle of Man
Flying: http://www.dylansmith.net
Oolite-Linux: an Elite tribute: http://oolite-linux.berlios.de
Frontier Elite Universe: http://www.alioth.net

[Andrew Sarangan]

Clustering can reduce the rate for one year and elevate it for another
year. However, the report shows that accident rates have been declining
for the past 10 years, which is a significant point.


Jose wrote:
> > I read on AvWeb today the '05 Nall Report is out, and apparently 2004
> > was a historic low for aviation accidents. This led me to question what
> > next year's report will look like after considering the spate of GA
> > accidents that we've all read about in the last year or so. Maybe it's
> > just a question of perception? Or, are pilots just getting too
> > complacent when they strap into their aircraft?
>
> Or maybe it's just a statistical artifact. Shift a few accidents from
> December to January, and shift a few others from next January to this
> December, and you have a banner year for airplane crashes caused simply
> by the artificial boundaries of the sample set.
>
> Sometimes random events cluster for no reason. In fact, it is highly
> unlikely that they would =not= cluster.
>
> Jose
> --
> Money: what you need when you run out of brains.
> for Email, make the obvious change in the address.

[[email protected]]

Dylan wrote:
>>>One or two a week? There are almost always at least a dozen GA accidents
a week.<<<

I realize this, I guess I was just referring to the accidents that got
local or national attention.

>>>You're probably only percieving a cluster because they have been talked
about more, but really - the accidents that have been talked about here

recently are not out of the ordinary (sadly). <<<

Yeah, I mentioned it probably being a question of my own perception
because of the ongoing discussion on this forum of recent accidents.
And yes, it is sad.

[[email protected]]

Andrew wrote:
>>>>Clustering can reduce the rate for one year and elevate it for another
year.<<<<

Right, Jose mentioned this in his reply to my OP

>>>>However, the report shows that accident rates have been declining
for the past 10 years, which is a significant point<<<<

That IS good news. I'm curious to see how/if '05 follows to the overall
trend.

[Skylune]

by Jose > Mar 9, 2006 at 09:40 PM




I haven't read the study so I can't really comment on it intelligently.
But like the difference between precision and accuracy, there is a
difference between pro-rated costs and marginal costs which needs to be
taken into account. One must also consider who benefits - it's not just
the immediate users.

And even if xxx were subsidized, that's not necessarily a bad thing.

Jose

<<

True. Not NECESSARILY a bad thing. I agree. That means the subsidies
must stand up to the political/budgetary debate about why they are
desirable, in the context of competing budgetary demands.

I disagree with those who claim there are no subsidies -- the data, even
if imperfect, makes this plain. See the BTS and FAA websites that track
amount of moneys deposited into the Aviation Trust Fund by source- you
will see that AV gas taxes are a tiny, tiny percentage.

[Matt Barrow]

"Andrew Sarangan" > wrote in message
oups.com...
> Clustering can reduce the rate for one year and elevate it for another
> year. However, the report shows that accident rates have been declining
> for the past 10 years, which is a significant point.
>
>
> Jose wrote:
>> > I read on AvWeb today the '05 Nall Report is out, and apparently 2004
>> > was a historic low for aviation accidents. This led me to question what
>> > next year's report will look like after considering the spate of GA
>> > accidents that we've all read about in the last year or so. Maybe it's
>> > just a question of perception? Or, are pilots just getting too
>> > complacent when they strap into their aircraft?
>>
>> Or maybe it's just a statistical artifact. Shift a few accidents from
>> December to January, and shift a few others from next January to this
>> December, and you have a banner year for airplane crashes caused simply
>> by the artificial boundaries of the sample set.

"Shift" as in reporting a December accident as if it occured in January?

[Capt. Geoffrey Thorpe]

> wrote in message
ups.com...
> Dylan wrote:
>>>>One or two a week? There are almost always at least a dozen GA accidents
> a week.<<<
>
> I realize this, I guess I was just referring to the accidents that got
> local or national attention.

Setting aside the smart ass mode for a moment or so. I think what hapens is
"event A" gets wide spread attention (for whatever reason) so, when similar
event B occurs, it gets attention since A was obviously newsworthy, then C
and D get noticed and the whole thing snowballs.

>>>>You're probably only percieving a cluster because they have been talked
> about more, but really - the accidents that have been talked about here
>
> recently are not out of the ordinary (sadly). <<<
>
> Yeah, I mentioned it probably being a question of my own perception
> because of the ongoing discussion on this forum of recent accidents.
> And yes, it is sad.
>

Indeed.

And, there are times it bothers me to be working for a car company. I
suspect that the automobile has turned out to be the deadliest weapon ever
invented by man - in spite of how hard we try to make it othewise...

--
Geoff
The Sea Hawk at Wow Way d0t Com
remove spaces and make the obvious substitutions to reply by mail
Spell checking is left as an excercise for the reader.

[Jose]

> "Shift" as in reporting a December accident as if it occured in January?

No, shift as having an accident in December instead of January.

Jose
--
Money: what you need when you run out of brains.
for Email, make the obvious change in the address.