• Welcome to the Speedsolving.com, home of the web's largest puzzle community!
    You are currently viewing our forum as a guest which gives you limited access to join discussions and access our other features.

    Registration is fast, simple and absolutely free so please, join our community of 40,000+ people from around the world today!

    If you are already a member, simply login to hide this message and begin participating in the community!

[Official] [Disallowed WR] Max Park - 6.44 3x3OH Single

xyzzy

Member
Joined
Dec 24, 2015
Messages
2,876
Misscramble found during competition ⇒ extra scramble.
Misscramble found after ⇒ keep the result.

Not that difficult. In Feliks's case, it was a suspected misscramble, but the delegate decided not to give an extra attempt for whatever reason and the misscrambling was only confirmed after the competition ended.

I think the real solution here is to carefully check every scramble every world-class cuber gets. Not just the world/continental records, not just the good solves, but all of them. Many of them record their solves and some competitions are live-streamed; look through the footage and check everything. Some people have pointed out that only good solves receive scrutiny, and this leads to WRs/whatever being retracted while less-notable solves get a pass (e.g. qqwref's post on r/cubers); this is a broken process and is unfairly biased against good cubers. If good solves that were misscrambles get retracted and replaced with likely-worse solves, but bad solves that were misscrambles don't get replaced with likely-better solves, that's effectively penalising competitors for factors outside of their control.

(This is assuming misscrambles lead to scrambles of about the same difficulty—an assumption that mostly holds, but not perfectly. There's a rough intuitive reason that random-state misscrambles are actually slightly easier on average: if (say) a clockwise turn was replaced by an anticlockwise turn in the scramble, there can be a shorter "locally optimal" 2-phase solution but not a longer one. The 2-phase solution length doesn't correlate much with scramble difficulty wrt CFOP, Roux and other human methods, though.)

Edit: On second thought, if checking all of the good cubers' solves is too much work (I mean, this could easily run into hundreds of solves to check per week), a reasonable-sounding compromise solution could be to also check the worst solve every time the best solve (which is possibly a record) is checked.

On third thought, this would be a useless compromise. The solves that are checked would be in like the top percentile, and conditioned on the best solve being at the 99th percentile, the worst solve is expected to be around the 20th percentile. Let's say the best solve does indeed turn out to be a misscramble. Since the probabilities of the different solves being misscrambles are independent, the probability of the worst solve being a misscramble as well is the same as the base rate of misscrambles (say, around 1%). Conditioned on the best solve being replaced, (i) the probability of the worst solve being replaced is just P(misscramble) ~ 1%, and (ii) the worst solve is less bad than the best solve is good, i.e. improving the worst solve is not a given, while worsening the best solve essentially is.

To introduce symmetry between the best/worst times, thereby (provably) removing the current bias against good results, we could unconditionally check the best and worst times in every ao5 by all the top cubers. Again, for fairness reasons, this should be regardless of whether the best time is record-worthy or whether the worst time is due to a DNF. This checks only 2/5 solves rather than 5/5 as I initially suggested, but fails to reliably catch situations where the top two solves are both notable (e.g. both sub-5) and the second best is a misscramble.

Yet another problem is that checking the results of only top cubers will fail to cover relatively unknown people (e.g. Steve Cho), but it's hard to do much about this.
 
Last edited:
Joined
Jun 30, 2015
Messages
1,320
Location
Brisbane, Australia
WCA
2015PEAR02
YouTube
Visit Channel
Feliks' 6.88 was a misscramble, yes, though his was something like an R' instead of an R. Even if the person scrambled it correctly, it (the time/reconstruction) wouldn't have been that far off.
I’m fairly certain this is the same as Max’s case, there was 1 or 2 moves done incorrectly towards the end of the scramble (saw the recon somewhere). It isn’t the reason this solve was denied
 

turtwig

Member
Joined
Apr 2, 2015
Messages
656
iirc There was an extra L move at the end (and Max coincidentally did an L' at the start of his solve).

The only difference I see between Max's and Feliks's cases are the Feliks's misscramble was only found after the comp. Honestly I think that Feliks's solve should have just been DNF'd. It would be a small disadvantage for Feliks but to me it doesn't make sense that his misscramble is more legit than Max's just because he couldn't get a replacement solve. In any case, the decisions for these two solves should be the same (either both accepted or neither).
 

ARobey

Member
Joined
Jun 7, 2018
Messages
35
Location
Iowa
WCA
2014ROBE05
Misscramble found during competition ⇒ extra scramble.
Misscramble found after ⇒ keep the result.

Not that difficult. In Feliks's case, it was a suspected misscramble, but the delegate decided not to give an extra attempt for whatever reason and the misscrambling was only confirmed after the competition ended.

I think the real solution here is to carefully check every scramble every world-class cuber gets. Not just the world/continental records, not just the good solves, but all of them. Many of them record their solves and some competitions are live-streamed; look through the footage and check everything. Some people have pointed out that only good solves receive scrutiny, and this leads to WRs/whatever being retracted while less-notable solves get a pass (e.g. qqwref's post on r/cubers); this is a broken process and is unfairly biased against good cubers. If good solves that were misscrambles get retracted and replaced with likely-worse solves, but bad solves that were misscrambles don't get replaced with likely-better solves, that's effectively penalising competitors for factors outside of their control.

(This is assuming misscrambles lead to scrambles of about the same difficulty—an assumption that mostly holds, but not perfectly. There's a rough intuitive reason that random-state misscrambles are actually slightly easier on average: if (say) a clockwise turn was replaced by an anticlockwise turn in the scramble, there can be a shorter "locally optimal" 2-phase solution but not a longer one. The 2-phase solution length doesn't correlate much with scramble difficulty wrt CFOP, Roux and other human methods, though.)

Edit: On second thought, if checking all of the good cubers' solves is too much work (I mean, this could easily run into hundreds of solves to check per week), a reasonable-sounding compromise solution could be to also check the worst solve every time the best solve (which is possibly a record) is checked.

On third thought, this would be a useless compromise. The solves that are checked would be in like the top percentile, and conditioned on the best solve being at the 99th percentile, the worst solve is expected to be around the 20th percentile. Let's say the best solve does indeed turn out to be a misscramble. Since the probabilities of the different solves being misscrambles are independent, the probability of the worst solve being a misscramble as well is the same as the base rate of misscrambles (say, around 1%). Conditioned on the best solve being replaced, (i) the probability of the worst solve being replaced is just P(misscramble) ~ 1%, and (ii) the worst solve is less bad than the best solve is good, i.e. improving the worst solve is not a given, while worsening the best solve essentially is.

To introduce symmetry between the best/worst times, thereby (provably) removing the current bias against good results, we could unconditionally check the best and worst times in every ao5 by all the top cubers. Again, for fairness reasons, this should be regardless of whether the best time is record-worthy or whether the worst time is due to a DNF. This checks only 2/5 solves rather than 5/5 as I initially suggested, but fails to reliably catch situations where the top two solves are both notable (e.g. both sub-5) and the second best is a misscramble.

Yet another problem is that checking the results of only top cubers will fail to cover relatively unknown people (e.g. Steve Cho), but it's hard to do much about this.
But what is the likelihood of a misscramble? The reason you should only check the fast solves is because with the misscramble, it is likely that it was easier to solve, therefore a faster time. It really is not that likely to get a misscramble, so why go to so much trouble just to check more scrambles and take more time out of a competition.

Also, people who are faster are also more likely to have a scrambler set up an easy position, so when fast people get a world record, it is more likely that scramble was set up rather than if you got a PB, therefore needing to be checked.
 

xyzzy

Member
Joined
Dec 24, 2015
Messages
2,876
But what is the likelihood of a misscramble? The reason you should only check the fast solves is because with the misscramble, it is likely that it was easier to solve, therefore a faster time. It really is not that likely to get a misscramble, so why go to so much trouble just to check more scrambles and take more time out of a competition.
I've done 30-ish reconstructions of official solves and 2 of them were misscrambles. They're rarer in well-run competitions with experienced scramblers (or at least I hope so!), but they still happen often enough that we have to care about it. You don't even have to dig very far to get examples: there's this and there's Jeff's 3BLD mean WR.

Also, I'm pretty sure I explained my point well enough: the current system is biased against good cubers. Consider the following.

(i) We check good solves. If they're misscrambles, they get replaced with likely-worse times.
(ii) We check bad solves. If they're misscrambles, they get replaced with likely-better times.

If we do (i), why don't we do (ii) as well? (ii) has obvious implementation issues—how bad does a bad solve have to be to qualify for being checked? Do we allow competitors to use their own video footage to check the scrambles? What if they find that they had multiple misscrambles for a round and they report only the ones that are likely to improve their times?

I'm not saying it's a good idea to check everything. It obviously slows the competition down a lot. I'm just saying it's the only truly unbiased procedure. (The procedure matters! That's the whole point of the discussion!) We can make compromises somewhere, but sticking to only (i) is clearly unfair and should change at some point.

(I think Chris Tran is working on some scramble checking automation, judging from his latest video. I assume he talked more about it in his seminar. Does @4Chan have any words to add here…?)
Also, people who are faster are also more likely to have a scrambler set up an easy position, so when fast people get a world record, it is more likely that scramble was set up rather than if you got a PB, therefore needing to be checked.
Goddamn, just disqualify every kid who does this. It's already in the regs that the scrambler can be disqualified for this.
 

One Wheel

Member
Joined
Feb 24, 2016
Messages
2,883
Location
Wisconsin
WCA
2016BAIR04
If good solves that were misscrambles get retracted and replaced with likely-worse solves, but bad solves that were misscrambles don't get replaced with likely-better solves, that's effectively penalising competitors for factors outside of their control.

You've clearly put a lot more thought into this than I have, but for what it's worth: good solves are likely to be much closer to optimal solves than bad ones. In other words, what scramble is used matters for a 5-second solve, but really doesn't for a 30-second solve.
 

ARobey

Member
Joined
Jun 7, 2018
Messages
35
Location
Iowa
WCA
2014ROBE05
I've done 30-ish reconstructions of official solves and 2 of them were misscrambles. They're rarer in well-run competitions with experienced scramblers (or at least I hope so!), but they still happen often enough that we have to care about it. You don't even have to dig very far to get examples: there's this and there's Jeff's 3BLD mean WR.
But 2 out of 30 solves is only 6.6 percent. In an average of 5, It is highly likely that there will not be a misscramble. There is a maximum of 4 rounds per competition, which means 20 solves at most for one event. These odds suggest there will only be 1 misscramble out of those 20 solves. If there is a misscramble, It would only be worth taking the time to check the fast solves, otherwise you have to go through every solve to find that 1 misscramble. In well run competitions with experienced scramblers checking the scrambles, it is probably much lower, therefore reducing those odds, making it less likely for a misscramble.
 
Top