The US Team selection process in future years

Some good ideas on how to balance objective and subjective inputs. Worth considering.

As someone who tilts to the analytical I find myself drawn to the precision of mathematical decision frameworks. Mathematical solutions are precise, transparent and not prone to things like block voting.

However, I'd observe that when it comes to calculating our way to a WGC team there is a difference between precision and accuracy. Just because we can calculate something to three decimal places doesn't mean that the precision signifies anything particularly accurate about the mix of the many personal attributes that makes someone a viable contender to be world champion.

We calculate UST scores in part based on performance at prior Nationals in each class. Given the inevitable random pilot performance and luck factors, this has two effects. First, people who have the personal flexibility to put in the travel to attend multiple sequential Nationals in a class have a better shot and second, people who have the personal flexibility to fly in Nationals in multiple classes get multiple bites at the apple. We only count the best scores, not the duds, so flying more contests ups your chances -at least if you are a good enough pilot to score near the top some of the time. There are a handful of pilots who get good scores based on only two qualifying contests flown - they are some of the best and most consistent pilots I know. Personally I look at both the average score across ALL contests flown AND the top two scores when attempting to judge a pilot's skill level and consistency. Neither is perfect by itself.

Also, we count, small, short and devalued Nationals the same as long competitive ones - even Nationals that are three days and get a 5% discount for PRL purposes get 100% credit for UST purposes - at least as I read it. We give credit for WGC performance and Continental contests in equal proportion even though the former is likely more competitive than a typical US Nationals and the latter could be the same or less - depending on participation. Adding these contests give pilots yet more scores to put into the mix and the WGC number gets counted across multiple classes, not just the one flown. More bites at the apple.

I'm not saying I necessarily disagree with the way things are calculated for the most part, but I would caution that just because you can calculate a score to three decimal places doesn't mean that those decimal places tell you something that's very accurate. How you arrange the formula has a big impact.

IMO a system that adds in peer review voting in a way that limits movement up or down the list to two or three places would be appealing. (BTW, it's my understanding that this is a primary purpose of the UST committee - to weed out voting monkey business). Tim's suggestion that we use a 1-10 scale where pilots in contention would typically earn a 9 or 10 seems like an opportunity for a small number of ill intentioned voters to blackball a pilot by giving them a 1 or 2. It's a good idea in general, but probably not implemented exactly that way.

9B



On Thursday, November 9, 2017 at 11:57:21 AM UTC-5, Tim Taylor wrote:
The system selection process could be improved by utilizing a hybrid system of objective (hard numbers from contests) with subjective (pilot votes). The current system used objective values to select those eligible but then used subjective voting to create the final list.

The forced choice ranking was subject to large variance and the absolute values were used to rank the pilots without looking to see if they were statistically significantly different. Rather than forced choice ranking a value of 0 to 10 could be given to each pilot during the voting. Remember all of these pilots were above 88 percent to make the list. Most should be getting an equavelnt score between 9 and 10 in voting.

On Thursday, November 9, 2017 at 8:08:26 PM UTC-5, Andy Blackburn wrote:
Some good ideas on how to balance objective and subjective inputs. Worth considering.

As someone who tilts to the analytical I find myself drawn to the precision of mathematical decision frameworks. Mathematical solutions are precise, transparent and not prone to things like block voting.

However, I'd observe that when it comes to calculating our way to a WGC team there is a difference between precision and accuracy. Just because we can calculate something to three decimal places doesn't mean that the precision signifies anything particularly accurate about the mix of the many personal attributes that makes someone a viable contender to be world champion.

We calculate UST scores in part based on performance at prior Nationals in each class. Given the inevitable random pilot performance and luck factors, this has two effects. First, people who have the personal flexibility to put in the travel to attend multiple sequential Nationals in a class have a better shot and second, people who have the personal flexibility to fly in Nationals in multiple classes get multiple bites at the apple. We only count the best scores, not the duds, so flying more contests ups your chances -at least if you are a good enough pilot to score near the top some of the time. There are a handful of pilots who get good scores based on only two qualifying contests flown - they are some of the best and most consistent pilots I know. Personally I look at both the average score across ALL contests flown AND the top two scores when attempting to judge a pilot's skill level and consistency. Neither is perfect by itself.

Also, we count, small, short and devalued Nationals the same as long competitive ones - even Nationals that are three days and get a 5% discount for PRL purposes get 100% credit for UST purposes - at least as I read it. We give credit for WGC performance and Continental contests in equal proportion even though the former is likely more competitive than a typical US Nationals and the latter could be the same or less - depending on participation. Adding these contests give pilots yet more scores to put into the mix and the WGC number gets counted across multiple classes, not just the one flown.. More bites at the apple.

I'm not saying I necessarily disagree with the way things are calculated for the most part, but I would caution that just because you can calculate a score to three decimal places doesn't mean that those decimal places tell you something that's very accurate. How you arrange the formula has a big impact.

IMO a system that adds in peer review voting in a way that limits movement up or down the list to two or three places would be appealing. (BTW, it's my understanding that this is a primary purpose of the UST committee - to weed out voting monkey business). Tim's suggestion that we use a 1-10 scale where pilots in contention would typically earn a 9 or 10 seems like an opportunity for a small number of ill intentioned voters to blackball a pilot by giving them a 1 or 2. It's a good idea in general, but probably not implemented exactly that way.

9B



On Thursday, November 9, 2017 at 11:57:21 AM UTC-5, Tim Taylor wrote:
The system selection process could be improved by utilizing a hybrid system of objective (hard numbers from contests) with subjective (pilot votes). The current system used objective values to select those eligible but then used subjective voting to create the final list.

The forced choice ranking was subject to large variance and the absolute values were used to rank the pilots without looking to see if they were statistically significantly different. Rather than forced choice ranking a value of 0 to 10 could be given to each pilot during the voting. Remember all of these pilots were above 88 percent to make the list. Most should be getting an equavelnt score between 9 and 10 in voting.


Forgot to mention the percentage "kickers" added to scores in world or continental competitions. It's another somewhat arbitrary number that has a significant impact.

9B

On Thursday, November 9, 2017 at 8:08:26 PM UTC-5, Andy Blackburn wrote:
IMO a system that adds in peer review voting in a way that limits movement up or down the list to two or three places would be appealing.

Along those lines, perhaps there could be a statistical trigger (e.g., based on standard deviation or other measures of variance) when there's unusual dispersion among the votes. Or when the voting order differs from the objective ranking by some amount. [Actually, higher dispersion within the voting should correlate most of the time with differences between the two rankings but I can envision situations (valid and questionable) where that would not be true.]

That trigger might call for closer scrutiny by the Team Committee, a second round of voting, a survey of the voters, or some other action or review to minimize questions about the validity of the selection.

I'm sure Andy could develop an algorithm that would serve as a trigger or circuit breaker or alarm for this purpose.

Chip Bearden

From the outside looking in:

Any time there is subjectivity someone will get upset, for that reason I prefer a straight analytical approach.
I had to fight to be on my first team because one of the excellent established team members had almost got a podium at the previous WGC, I would have reacted badly if he beat me on the team as I had beaten him two years in a row for the selection. (As it happens he didn't go anyway and I was the only Standard class entry.)
Team flying is incredibly difficult to do well and the pilots should be able to sort out what to do about whether to team fly or just cooperate.
I got on well with all the US team at Benalla, including both Seans, and saw no disharmony.
Re pilots not going once selected: many reasons, time, money, motivation. I was selected for 2018 in 18M (with our selection I would have needed to go to the pre-worlds or that spot is under threat this season) but time/work and motivation after four WGCs in a row have beaten me down, in a few years I might try again.
Money wise we probably get even less than the US team but I have been very fortunate to fly gliders owned by very generous people. (Thanks again Al )
The current team/gaggle flying as was so important at Benalla I find dangerous and only judges certain skills. If the IGC eventually changes a few starting and numbers rules so a "lone wolf" could win then I might go again, although flying in Europe in flatlands I find a downgrade compared to home (or Uvalde! )
Like Sean F I like fixed tasks but am happy with AATs about 50% of the time, I have no experience of MATs but expect it would be a bit of a lottery (that skilled pilots would get lucky with a lot). GP flying I find a lot of fun and sorry that work will stop me trying for the next World GP in Spain. ( the selection for World GPs is really easy!)
I have now flown five WGCs with two top tens and three mid fleet results, three day wins (with one at Uvalde and one in Leszno being AATs!)
Happy flying,
Tom Claffey (former Aussie team member)