The Statistical Case for Combined Finals

[qqwref]

LOL. We are discussing a format change that would potentially affect the rankings - are we not?

We are not. This would only affect the rankings in individual competitions, which will also affect who becomes a national/regional/world champion.

There is also the difference between overall WCA rankings, and the rankings for any single tournament event.

I know Kirjava well enough to be certain of what he meant in his post; this is one advantage of making friends on the forum. Either you've misunderstood his post, or you misinterpreted the nature of the discussion (see the above).

The key thing to realize is that the serious competitors of a major comp. are now too close in ability for any one 5-solve trimmed to decide it fairly.

Completely false. I watched the finals of Nationals 2010 and it was clear that only two or three competitors had the speed and calm to even be capable of matching or beating Rowe's average, and he was clearly better than they were. I have already stated that I think factors such as these exert much more of an effect than pure randomness along the lines of a normal distribution. Because the finals feature the same scrambles and very similar methods, this is even more the case than it would be in a general situation.

No ****Sherlock! What have I actually been suggesting as a nontrivial improvement for determining which result(s) should be considered better?

Do you know what I meant by "nontrivial"? The point of my 3rd sentence was that the question of which of those two competitors is better is not at all obvious (nontrivial, that is) and is more worthy of debate than you seem to think from your previous post. The alternatives (best round, final round, trimmed overall average, untrimmed overall average, average of averages) have been suggested and now we are on the stage of debating which provides better results.

Guys how can a discussion about statistics and competition format result in such tone?

When any participant is the type who might launch into a profanity-filled rant when his posts are criticized. Just sayin'.

This (scramble equity) is one of the key objections to using the best average for any round format, but this shouldn't be much of a problem with a trimmed running average.

This could be a good point, but - why not? People can still get different scrambles in almost all of the competition. Why are you concerned about an avg5 with all different scrambles, but not about an avg15 with 10 different scrambles? Anyway, I still don't think scrambles matter as much as human factors, unless at least two extremely easy scrambles pop up in one round.

If we use 15 solves to rank every competitor, how do we eliminate people into smaller and smaller rounds?

I imagine that you would need many different setups depending on the number of solves that have been done so far, whether it be 5, 10, 15, or something odd like 2 or 7. These trimmed averages of N would then be used for deciding who makes the next round. I think it's safe to say that 15 is just an example - not all competitions have 3 rounds, and if we're going to be doing an overall average we don't want to make up numbers or discard solves.

If the winner is decided by the best average, then it would seem that it would really take away from the entire concept of the finals.

Suppose Rowe goes up first in the finals and gets the best average so far in the competition. Now the situation is exactly the same as in the best-average-ever scenario, at least for everyone but Rowe. Would that take away from the concept of the finals?

If we used the best-average-ever condition, finals would mean the last opportunity to improve your rank in the competition. This would run parallel to the idea that the finals is the last opportunity to improve your rank in the world (which is how I treat it since I care more about the world rank than about the rank at any individual competition except Nationals or Worlds).

 

[reThinking the Cube]

Guys there is one other thing to bring up here. If we use 15 solves to rank every competitor, how do we eliminate people into smaller and smaller rounds?

Here is a format that can easily and fairly determine which competitors deserve to advance to the next round, but doesn't eliminate strong competitors early, AND be implemented without placing additional burden on organizers, uses all 15 solves to better decide the winner, generates official WCA rank times for each round, and is more spectator/competitor friendly than the current format.

All(3) rounds of 5 are done exactly the same as the current format. The 5-solve rounds are trimmed and averaged the same, to produce an official WCA time for that round for that event. Round#1 is 5 solves and averaged just like it is now.

So the question now is - Who gets to advance to Round#2?

The current format is to use the Round#1 times as the basis for elimination. This leaves open the possibility of a strong cuber getting more than 1 bad result in the first round and not advancing.

Rather than using this official time as the basis for deciding who advances, I really like the idea of temporarily double trimming out the 2 highest and 2 lowest times to give just the median time for that round. For all the tournaments that I have looked at, this median value has generated a fairer selection than using the official 5-solve round time. The main reason for this being the chance that a serious competitor could get eliminated in the first round by simply getting more than 1 bad solve. If the median value is used instead, it would take 3 bad solves out of 5 to prevent that competitor from advancing. So the competitors could easily be ranked(for cutoff placement) and eliminated based on this time. This median value is only used for determining advancement into Round#2 and is not needed for anything other than that.

Round#2 is also 5 solves and done exactly like it is now, and will produce another official WCA result for the competitors that get to do it.

So the question now is - Who gets to advance to Round#3?

The current format is to use the Round#2 times as the basis for elimination. This leaves open the possibility of a strong cuber(Sikan Li) getting more than 1 bad result in the second round and not advancing.

Rather than using this official time as the basis for deciding who advances, I really,really like the idea of temporarily trimming out the 3 highest and 3 lowest times for all 10 solves done so far, and then averaging the median 4 times that are left. For all the tournaments that I have looked at, this median average value has generated a much fairer selection than using just the official 5-solve round#2 time. The main reason for this being the chance that a serious competitor could get eliminated in the second round by simply getting more than 1 bad solve in that round. If this median average value is used instead, it would take 4 or more very bad solves out of 10 to prevent that competitor from advancing to the final round. So the competitors could easily be ranked(for cutoff placement) accordingly, and eliminated based on this time. This median value is only used for determining advancement into Round#3 and is not needed after that.

Round#3 is also 5 solves and done exactly like it is now, and will produce another official WCA result for the competitors that get to do it.

So the question now is - Who should win?

The current format is to use the Round#3 times as the sole basis for determining the winner. This leaves open the very real possibility of the strongest cubers getting more than 1 bad solve out of these last 5, and ruining their tournament placement because of it. Instead, I really,really,really like the idea of trimming out the 3 highest and 3 lowest times from all 15 solves, and then averaging the 9 median times that are left to produce a tournament time for that event. For all the tournaments that I have looked at, this value has produced by far, the fairest placement rankings. Trimming more or less than 3/3 times from the 15 is also possible, but doing it this way, keeps it consistent with the current 40% trim.

 

[cmhardw]

Guys there is one other thing to bring up here. If we use 15 solves to rank every competitor, how do we eliminate people into smaller and smaller rounds?

Here is a format that can easily and fairly determine which competitors deserve to advance to the next round, but doesn't eliminate strong competitors early, AND be implemented without placing additional burden on organizers, uses all 15 solves to better decide the winner, generates official WCA rank times for each round, and is more spectator/competitor friendly than the current format.

[method description]

Wow... I have to admit, that is a pretty cool way to do things. Perhaps a hybrid of this and the usual trimmed average of 5 could be used as well?

Yeah, I do like this idea. I think this, or slight variations of it, could stand a real chance of being an effective way to run a competition that is also (possibly?) more fair. Anyone else have any comments?

Chris

 

[Faz]

So, if this were to be implemented, this would force all competitions to have 3 rounds of 3x3 would it not? - Sometimes, due to lack of competitors, or time restrictions, this isn't possible. Out of 4 competitions I have been to, only one of them has had 3 rounds of 3x3. What about for other events? Would the winner of 4x4 be determined by the median time?

 

[hawkmp4]

Rather than using this official time as the basis for deciding who advances, I really,really like the idea of temporarily trimming out the 3 highest and 3 lowest times for all 10 solves done so far, and then averaging the median 4 times that are left. For all the tournaments that I have looked at, this median average value has generated a much fairer selection than using just the official 5-solve round#2 time. The main reason for this being the chance that a serious competitor could get eliminated in the second round by simply getting more than 1 bad solve in that round. If this median average value is used instead, it would take 4 or more very bad solves out of 10 to prevent that competitor from advancing to the final round. So the competitors could easily be ranked(for cutoff placement) accordingly, and eliminated based on this time. This median value is only used for determining advancement into Round#3 and is not needed after that.

Why trim the 3 lowest and 3 highest? Why not just one? Why not take the median of all the solves? It just seems arbitrary.
Also, how would you propose to run competitions where only 2 rounds of 3x3 are possible?

 

[qqwref]

Here is a format [...]

Round#1 [...] just the median time for that round. [...]
Round#2 [...] trimming out the 3 highest and 3 lowest times for all 10 solves done so far, and then averaging the median 4 times that are left. [...]
Round#3 [...] trimming out the 3 highest and 3 lowest times from all 15 solves, and then averaging the 9 median times that are left to produce a tournament time for that event.

OK, but why do you choose to do trimming in exactly this way? Doesn't it seem a bit arbitrary? And what would we do if we have 20 solves to choose from (4 rounds)? What if the first round is a qualification and there are only 2 solves there, so instead of 10 after the second round we have 7? (This has happened.)

What about for other events? Would the winner of 4x4 be determined by the median time?

Good point. I don't think a trimmed global average is a good way to do all events. For 4x4 or Square-1, for instance, if parity always gives you a slower time, you will then have to be lucky to get 3 non-parity solves rather than 2 (in which case your median will be a parity solve and thus slow). I'm just talking about the first round, but of course few comps will have a second.


Here is something interesting: I think that, as a cube gets bigger, the quality of a solve starts to matter a lot more than how easy it is. Essentially, there will be far more variation due to turnspeed, lookahead, lockups, tired wrists, and concentration than due to scramble difficulty or random case factors. So, for large cubes with multiple rounds in a competition, I think that taking the mean average or some kind of overall trimmed average will mostly measure how well someone holds up under pressure, whereas taking the fastest average will measure how well someone does in good circumstances.

 

[JeffDelucia]

I think that winners of event should just be decided by who already has the fastest official average because averages are a good representation of speed and whoever has the fastest speed should win!

Spoiler

rofl sarcasm

 

[RyanO]

The winner of a competition shouldn't necessarily be the best cuber. A competitive environment changes how you perform. The best cuber may not be the best competitor. If you don't perform well in the finals you shouldn't win the competition.

 

[cmhardw]

Good points about having 2 or 4 rounds for 3x3x3, as well as how things would be decided for bigger cubes. So far it seems that the consensus is that the current method, though also arbitrarily picked by the first organizers of competitions, is at least not worse than any of the other proposed methods when all factors are taken into account. If this is the case I would strongly favor not changing the rules unless a clearly superior method is discovered, as the effects of the change-over would be quite far reaching.

Chris

 

[reThinking the Cube]

So, if this were to be implemented, this would force all competitions to have 3 rounds of 3x3 would it not? - Sometimes, due to lack of competitors, or time restrictions, this isn't possible. Out of 4 competitions I have been to, only one of them has had 3 rounds of 3x3. What about for other events? Would the winner of 4x4 be determined by the median time?

Any number of rounds is easy, and done the same way. The median times are only used to determine who advances to the next round. As it turns out, the winner of the event is also the only competitor to advance into the hypothetical final+1 round. The only difference for the last round is that the total number of trims allowed will always be 2 for each 5 round solve completed to get there.

Example, for a 2 round event, it is easy to use the median gotten from round 1 to get the final group of competitors. Then after the second round was completed, the winner would be determined from using all 10 solves trimmed by removing 2 highest and 2 lowest, average the other 6. All Official WCA 5-solve round times are always still preserved. This will produce a better Round2 selection, and also a fairer determination of the winner of the event, since 10 solves are used, and the trims can be applied uniformly to the whole set.

A 4 round event would be done the same way by extension, it makes no difference how many rounds there are. If it was 1 round, then the normal 5-solve trimmed average would decide the winner. That also answers the 4x4x4 question. These are not problems at all.

 

[Pedro]

Agreed with Chris.
There's too much variation in number of rounds/solves to be able to achieve a consistent method.
And, as some others said, the best cuber is determined by the world ranking, not the result of a single competition. Who wins the competition is who performed better that day/round.

Recently, I've been to a comp (ABC Open 2010[/url) and, not that I'm bragging about it, but I was clearly the best cuber there. Turns out my cube popped once (I was using a OH cube) and I twisted a piece in the final round, getting an average of 17.xx, which was almost not enough for 3rd place.

Does that mean I wasn't the best cuber there? I don't think so. It means I performed bad in the last round, due to many factors.

I think competitions measure a different thing than the world rankings.
Within a competition, some people in some place are cubing together, and the best performer wins that comp.
But everyone's times go to the world ranking, which includes each competion that happened, no matter where. The best cuber, to me, is Feliks at the moment, because he has done the best performance ever.

 

[hawkmp4]

Okay, again, why do you want to trim the averages the way you do? It's arbitrary. Why trim 4 from the averages of 10? If you're looking for consistency, taking the median time is going to be less affected by outliers.

 

[qqwref]

Then after the second round was completed, the winner would be determined from using all 10 solves trimmed by removing 2 highest and 2 lowest, average the other 6. [...]

If it was 1 round, then the normal 5-solve trimmed average would decide the winner.

So you're changing your mind, then? You don't want to trim 2 on each side off a 5-solve average, and 3 on each side off a 10-solve average?

 

[Daniel Wu]

All of this is so convoluted that it would seem that the current system is better just because of the simplicity.

 

[reThinking the Cube]

Then after the second round was completed, the winner would be determined from using all 10 solves trimmed by removing 2 highest and 2 lowest, average the other 6. [...]

If it was 1 round, then the normal 5-solve trimmed average would decide the winner.

So you're changing your mind, then? You don't want to trim 2 on each side off a 5-solve average, and 3 on each side off a 10-solve average?

No change, it is very simple. All 5-solve rounds will always get 2 trims just like normal. Now, when there is another round to advance into, then you are given 2 extra trims (you could consider them to be borrowed in advance, from the upcoming 5-solve round) to be applied to ANY of the solves that have already been done in ANY previous round. The solve times that are left after trimming are then averaged, and this is the value that is used to determine who advances into that next round and that is all. This works perfectly no matter how many rounds there are. The final round cannot borrow, so that it trims at 2 per 5 solves, which gives the same trimming ratio as the current method.

be moar nice

 

[Faz]

All of this is so convoluted that it would seem that the current system is better just because of the simplicity.

This.

 

[aronpm]

I don't see any problems with the current system.

 

[reThinking the Cube]

I don't see any problems with the current system.

LOL. You're mostly blind anyway - right?

 

[qqwref]

No change, it is very simple.

You said in a previous post that you wanted to trim out four times for a single round, and six times for two rounds, to determine who would get into the next round. Now you say that you want to trim out two times per round, which would mean removing two for a single round and four for a double round. I cannot make this any clearer - your recommendation has changed.

If you're not intelligent enough to seriously contribute to the thread, you should probably just stop posting here.

 

[aronpm]

I don't see any problems with the current system.

LOL. You're mostly blind anyway - right?

LOL. You're mostly 4x4-edge-flip anyway - right?