[WCA Regulations 2014] Scramble Filtering (new poll)

[keyan]

It's unfortunate that the regulations were not immediately updated to reflect this, but I'm positive that was only because everyone involved is a volunteer and has other priorities, not because anyone was intentionally being secretive.

Unfortunately, whatever the reality, the appearance is that Sebastien wanted to, if not outright hide, at least bury this information in scrambler documentation that attracts much less community attention than the regulations.

I think that these limits should not appear in the regulations (4b3a-4b3c in the current draft). There is absolutely no use in that for anyone and for documentation, the TNoodle documentation seems absolutely sufficient to me.

https://groups.google.com/forum/#!topic/wca-scrambler/3x_hN9Y4HCM

Additionally:

we had agreed on filtering 0 and 1 moves from solved, or so i thought; then Sebastien says "Oh hey btw Board wants 4 for 2's and 8 for Pyra"
Tyson: We agreed on that? I wasn't aware we had discussion

The above is a direct quote from a conversation I had with a WRC member. I confirmed with Tyson that this was an accurate representation of the events prior to posting.

I don't like that we have regulations enacted unilaterally without proper consideration (see Thom's comment #75) and, if this (admittedly non-representative) poll is any indication, against the desires of the community.

 

[qqwref]

If you're suggesting that changes to TNoodle drove the board/WRC to change the regulations, I want it to be clear that that was very much not the case.

No, it's a different complaint entirely. I'm saying that any changes in TNoodle are effectively changes in the regulations (since scrambles are defined by what TNoodle does), but they nevertheless bypass the normal regulation process. The scrambles and method of scrambling can be (and are) changed without any change in the regulations as written, and separate from the normal regulation-change discussions and schedule.

 

[Lucas Garron]

No, it's a different complaint entirely. I'm saying that any changes in TNoodle are effectively changes in the regulations (since scrambles are defined by what TNoodle does), but they nevertheless bypass the normal regulation process. The scrambles and method of scrambling can be (and are) changed without any change in the regulations as written, and separate from the normal regulation-change discussions and schedule.

... which is why I insisted on making them part of the Regulations. Should something be changed about that?

 

[Dapianokid]

My only concern is that the people responsible for handling the inner workings behind TNoodle are basically responsible for large portions of the regulations.

The numbers behind the probablity of any kind of "bad" scrambling are abysmally small. That being said, the chances of "getting a last layer skip" are just as "unfair" to people as a 4 move 2x2 WR solution.
What if somebody came along and couold optimally solve puzzle near maximum depth?
A method is a method, and it solves the cube, and speedcubers like to do this fast. The "luck" they may get is part of that, and it's kinda depended on. But we still respect fast cubers even if they get what we consider "easy" scrambles (or we SHOULD respect them) in a competition becuase they are FAST. Nobody would be complaining if they all got "easy" scrambles. We wouldn't go so far to compete if we didn't at least want to get lucky, we'd just upload averages and be happy with what's out there.

I like the idea of filtered scrambles, and it's well implemented from what I can tell. I have read a lot of the debate and If anything, people should be happy that the playing field has been leveled out an almost unrealizably small amount by filtering.

 

[qqwref]

... which is why I insisted on making them part of the Regulations. Should something be changed about that?

No, insisting on making scrambling regulations part of the regulations is a good thing. I don't know why we stopped doing that.

 

[Kirjava]

people should be happy that the playing field has been leveled out an almost unrealizably small amount by filtering.

1 in 12 is not small.

L4E skip vs L4E is not level.

It is not possible to 'level the playing field'.

 

[Sebastien]

Unfortunately, whatever the reality, the appearance is that Sebastien wanted to, if not outright hide, at least bury this information in scrambler documentation that attracts much less community attention than the regulations.

It is always the easiest to assume bad intentions, right?

Here are some true alternatives:

- leaving out information that well just confuse not so experienced WCA members, seen how many problems we already have getting those to read our regulations.
- leaving TNoodle and the regulations as disconnected as possible to not complicate changes. As a matter of fact, the Board has already decided to change the limits for Pyraminx and Skewb down to 6 moves, based on the community's feedback in this thread. We cannot make this change in TNoodle 0.8.0 (which will be released at the beginning of next week) though, because we would violate the regulations otherwise.

Additionally:

we had agreed on filtering 0 and 1 moves from solved, or so i thought; then Sebastien says "Oh hey btw Board wants 4 for 2's and 8 for Pyra"
Tyson: We agreed on that? I wasn't aware we had discussion

Me: I want to know where this conversation should have happened
chris: I was told second hand, so I don't know that.

This is just defamatory on the lowest possible level. I managed to find the refered conversation in my mail archive and what Chris presents here as quote is not even roughly corresponding to that conversation's actual content. I forwarded the original conversation to the WDC.

 

[tim]

Imagine a scramble that is widely seen as 'easy', but has a sufficiently long optimal solve to be valid under preemptive filtering. For example, superflip. Consider the response to the appearance of such a scramble at competitions in two parallel universes, one where scramble filtering occurs and another where it does not.

In the universe where there is no scramble filtering, the response need only be 'All cube states are potential scrambles, and everyone has an equal chance of encountering scramble states of varying subjective difficulty.' There needn't be controversy, as all scramble states are treated equally.

In the universe where scramble filtering does occur, the response is likely more complex. 'By filtering some scrambles, we presuppose that there are some scramble states that we find unacceptable. However, there are many, many more states that some cubers may be unhappy to see occur. Eliminating scrambles based on optimal solve length attempts to address these 'easy scrambles', but it is impossible to satisfy everyone. Any attempt to eliminate these cases will leave some people unhappy.'

Optimal solve length filtering is far from eliminating cases that would commonly be seen as trivial. It is impossible to define an exhaustive set of 'too easy' scrambles that all cubers would agree with. Scramble filtering serves to cause controversy by acknowledging that there are some scramble states that are unacceptable, yet being unable to prevent some of them from being encountered. I'd rather our position be that all scramble states are acceptable, than that some are unacceptable but unavoidable.

I think this is a very strong argument against scramble filtering. We have created this artificial world in which some states are considered "acceptable" while others (potentially just as fast to solve) aren't. This is inconsistent, leads to longer regulations and seemed to be a headache for TNoodle's developers (at least for some puzzles if I interpreted the discussions correctly).

In my opinion no filtering would be the best solution for everyone involved. We as a community should just accept (fortunately most of us already are) that single solves can be heavily influenced by luck.

There's also the github repository here https://github.com/cubing/tnoodle/, which I would love to see more members of the community contribute to.

I think it would help attracting more developers if TNoodle wouldn't be that huge. Looking at the Github page is very intimidating when you except just a simple scramble generator. There are a dozen projects crammed into that repository which don't seem even slightly related to each other (cubecomps*, git-tools, quercus**, a timer, ...). What do you guys do if you have several issues related to different projects? Tag them differently? Prefix the titles? In my opinion most of these projects (if not all) should be moved to separate repositories.

* hasn't been updated for 9 months, so it's just a copy of some old source code?
** it's empty.

Don't get me wrong: I have huge respect for the work you guys have done in the last couple of weeks (have been reading every mail) and everyone involved seems to know exactly what he's doing. It's just the project organization which seems a bit off in my opinion and might discourage new developers taking part.

 

[Kirjava]

the Board has already decided to change the limits for Pyraminx and Skewb down to 6 moves, based on the community's feedback in this thread

When you say 'the Board' you mean solely you, right? Changing Skewb from 7 to 6 on what is essentially a whim reeks of a lack of justifiability for this. You say it's based on community feedback, but that's misleading because it's actually because you fu​cked up and thought the WR scramble was 7 moves and not 6.

 

[Tim Major]

When you say 'the Board' you mean solely you, right? Changing Skewb from 7 to 6 on what is essentially a whim reeks of a lack of justifiability for this. You say it's based on community feedback, but that's misleading because it's actually because you fu​cked up and thought the WR scramble was 7 moves and not 6.

And most of the "community" didn't say they wanted 6 moves. They said it should be at MOST 6 moves. It's like Sebastien is trying to compromise compared to most of the other posters in this thread.

 

[JustinJ]

As a matter of fact, the Board has already decided to change the limits for Pyraminx and Skewb down to 6 moves, based on the community's feedback in this thread.

I still don't quite understand what the motivation is for making it so high. While I am against scramble filtering in general, this seems to me not to line up with the original arguments made in favour of filtering (eliminating trivial cases). I'd say a 5-6 move scramble is certainly *easy*, but definitely not *trivial* on the level of a 2-3 move 2x2 scramble.

 

[tim]

Another argument against filtering: No one would waste his time arguing about (seemingly) arbitrary limits.

 

[jfly]

No, it's a different complaint entirely. I'm saying that any changes in TNoodle are effectively changes in the regulations (since scrambles are defined by what TNoodle does), but they nevertheless bypass the normal regulation process. The scrambles and method of scrambling can be (and are) changed without any change in the regulations as written, and separate from the normal regulation-change discussions and schedule.

Sorry for getting pedantic here, but I would like to distinguish between someone like me making random changes in TNoodle and the official TNoodle release process. Neither I nor anybody on the wca-scrambler team has control over this page: https://www.worldcubeassociation.org/regulations/scrambles/. The version of TNoodle pointed to on that page is the only version of TNoodle that may be used for competitions (anything else must be explicitly approved by the board, which has happened so far for every competition in 2014 with skewb). I hope that in the future, as the WCA recommends new official scrambler programs (new version of TNoodle, or something else entirely, who knows), that those programs will either follow the regulations, or the regulations will be changed to reflect the behavior of those programs. Again, everyone involved is a volunteer, and mistakes happen, especially when we're all ignoring our families during the holidays and rushing to finish stuff by an arbitrary deadline. I should have pushed back harder when I knew (admittedly a while ago) that TNoodle was not going to be ready for the 2014 regulations by the new year. Nobody involved wants TNoodle to diverge from what the regulations ask for.

I think it would help attracting more developers if TNoodle wouldn't be that huge. Looking at the Github page is very intimidating when you except just a simple scramble generator. There are a dozen projects crammed into that repository which don't seem even slightly related to each other (cubecomps*, git-tools, quercus**, a timer, ...). What do you guys do if you have several issues related to different projects? Tag them differently? Prefix the titles? In my opinion most of these projects (if not all) should be moved to separate repositories.

* hasn't been updated for 9 months, so it's just a copy of some old source code?
** it's empty.

Don't get me wrong: I have huge respect for the work you guys have done in the last couple of weeks (have been reading every mail) and everyone involved seems to know exactly what he's doing. It's just the project organization which seems a bit off in my opinion and might discourage new developers taking part.

You're right. For a long time, Lucas has been trying to convince me of how important this was, and I've only recently started to realize this. Keep in mind that when I started TNoodle, it was just a personal project, and I had no intention of it becoming the official wca scrambler. There are a lot of vestigial projects in there, some of which should be removed from the face of the earth, and others which should be moved to separate repositories. It is just a lot easier to create a few more files as opposed to creating a new repository and setting up git modules or git subtrees, or copying code or binaries around (all of which have their own special downsides).

It's probably not so important to talk about how TNoodle go to be the way it is. I am pretty proud of the current state of the code, but as you pointed out, the structure can be something of a mess. I also very much regret that we don't use a standard build system (it was originally Ant, and I moved away from it when it just didn't let me do what I needed to). The fact of the matter is that stuff does work the way it is right now, and while I would like to switch to a standard build system (gradle looks interesting), I don't have enough experience with anything else (I know I hate Ant, and we use automake and autoconf at my job) to make a good decision about where TNoodle should go next. I would like to have a discussion about possibilities on the wca-scrambler group, and hopefully come to a consensus about what to do. I think for any such discussion to be productive, someone is going to have to start out with a concrete suggestion, rather than a simple "stuff kind of sucks right now, we should make it better". If you have any ideas, please feel free to get the ball rolling by sending out an email to the wca-scrambler group.

Upon proofreading this post, I realize I've ignored all of your questions, if you really did want answers to any of them, let me know and I will answer as best I can.

 

[Sebastien]

When you say 'the Board' you mean solely you, right?

This is just getting ridiculous. If I say that "the Board" has decided something, that means that the majority of the Board members has agreed on a specific outcome. Your assumption that I'm abusing this term is, again, a personal insult. And I'm seriously fed up of being targeted personally, just because I'm the only board member who is stupid enough to face the angry mob on this plattform.

PS: Your assumption that the filter limit of 7 moves for Pyraminx was based on Odder's WR is simply wrong. This decision is even older than Odder's solve and based on Oka's 1.93 (having less moves than Bruno's 1.61) and was implemented in the first TNoodle release at the beginning of 2013 already. As I learned from Jeremy way later, the first implementation for Pyraminx filtering has been faulty, otherwise Odder's scramble would have never been generated. Have your fun blaming me for not knowing the move count of our current Pyraminx single WR, but I'm just one of many and at least I stepped up immidiately after getting to know about it to fix this inconsistency.

 

[XTowncuber]

6 moves is still too many, but it's better than 7.

 

[Kirjava]

This is just getting ridiculous. If I say that "the Board" has decided something, that means that the majority of the Board members has agreed on a specific outcome. Your assumption that I'm abusing this term is, again, a personal insult. And I'm seriously fed up of being targeted personally, just because I'm the only board member who is stupid enough to face the angry mob on this plattform.

The majority of the WCA board ok'd 7 moves without researching anything? No one checked?

PS: Your assumption that the filter limit of 7 moves for Pyraminx was based on Odder's WR is simply wrong.

At no point did I say that you based the limit of 7 on odder's world record. You are outright lying.

I said you changed the limit based on his world record, I have no idea where you initially pulled 7 from.

This decision is even older than Odder's solve and based on Oka's 1.93 (having less moves than Bruno's 1.61) and was implemented in the first TNoodle release at the beginning of 2013 already. As I learned from Jeremy way later, the first implementation for Pyraminx filtering has been faulty, otherwise Odder's scramble would have never been generated. Have your fun blaming me for not knowing the move count of our current Pyraminx single WR, but I'm just one of many and at least I stepped up immidiately after getting to know about it to fix this inconsistency.

So the only reason the limit is 6 is because of a programming error. This is just nuts.

 

[tim]

It's probably not so important to talk about how TNoodle go to be the way it is. I am pretty proud of the current state of the code, but as you pointed out, the structure can be something of a mess. I also very much regret that we don't use a standard build system (it was originally Ant, and I moved away from it when it just didn't let me do what I needed to). The fact of the matter is that stuff does work the way it is right now, and while I would like to switch to a standard build system (gradle looks interesting), I don't have enough experience with anything else (I know I hate Ant, and we use automake and autoconf at my job) to make a good decision about where TNoodle should go next. I would like to have a discussion about possibilities on the wca-scrambler group, and hopefully come to a consensus about what to do. I think for any such discussion to be productive, someone is going to have to start out with a concrete suggestion, rather than a simple "stuff kind of sucks right now, we should make it better". If you have any ideas, please feel free to get the ball rolling by sending out an email to the wca-scrambler group.

Thanks for your answer! Unfortunately I've no experience with the Java build ecosystem either, but know the language pretty well.

Since I hate these messy mailing lists/google groups, I've opened an issue on GitHub to get the discussion started: https://github.com/cubing/tnoodle/issues/160 Everyone seriously involved with the project should be on there anyway.

Upon proofreading this post, I realize I've ignored all of your questions, if you really did want answers to any of them, let me know and I will answer as best I can.

They were just questions floating around in my head when looking at the repository. Their answers don't matter.

\edit: Reading your response more carefully, I noticed you pretty much insisted on using the wca-scrambler mailing list. Sorry. Maybe you could link to the GitHub issue in a post?

 

[qqwref]

The version of TNoodle pointed to on that page is the only version of TNoodle that may be used for competitions

That doesn't help me out at all. I'm not a developer on TNoodle and I certainly haven't been following the discussions enough to know the difference between version 1.7.12 and any other versions. And because of that, all I see is, TNoodle is the official scrambler and you guys are doing your own development on it, perpendicular to regulations.

Also, while it's true the regulations specify some info about what a scramble looks like (e.g. which puzzles are purely random state, what the notation is) (although they didn't in the version used for the last ~6 months of 2013), TNoodle is still the ONLY allowed scrambler. If it scrambles in a way that's inconsistent with the regulations, either you can't generate scrambles, or you are effectively following TNoodle's definition rather then the regulations' one.

Again, everyone involved is a volunteer, and mistakes happen, especially when we're all ignoring our families during the holidays and rushing to finish stuff by an arbitrary deadline.

Here's the thing. From my perspective, most of the changes in the actual scrambling algorithms that I've seen in the last months (and I may have missed some) resulted in an ending product that was either worse, or different without being better. Our scrambling methods were good enough a year ago. As for changes not having to do with scrambling algorithms, I don't see why they are required for regulation compliance, as opposed to being stuff you guys wanted to do to be helpful. I'm sorry that you spent your free time this way instead of being with your families, but we didn't ask or demand for you all to do it.

It's the same thing I was thinking about the regulations group - they're putting in a lot of time, but ending up mostly with things the community doesn't care about or really doesn't want. I don't want you guys to work harder or put in more time - if anything, I want you to work better, and look before you leap.

 

[jfly]

They were just questions floating around in my head when looking at the repository. Their answers don't matter.

\edit: Reading your response more carefully, I noticed you pretty much insisted on using the wca-scrambler mailing list. Sorry. Maybe you could link to the GitHub issue in a post?

Ok, that's what I thought (about your questions).

While I do really like having everything in my inbox (easier for me to search for), github issues are fine. I don't have a good sense of when something is better suited for one or the other.

 

[jfly]

That doesn't help me out at all. I'm not a developer on TNoodle and I certainly haven't been following the discussions enough to know the difference between version 1.7.12 and any other versions. And because of that, all I see is, TNoodle is the official scrambler and you guys are doing your own development on it, perpendicular to regulations.

I'm not proud of the current state of TNoodle's documentation, but it's certainly not intentionally obscured. Again, we're volunteers, and when we get something working, that's when we stop and catch up on all the other stuff we've put on hold to get it done. I understand that you might believe that the things we're doing are not necessary or even worse, and I'll get to that later.

Also, while it's true the regulations specify some info about what a scramble looks like (e.g. which puzzles are purely random state, what the notation is) (although they didn't in the version used for the last ~6 months of 2013), TNoodle is still the ONLY allowed scrambler. If it scrambles in a way that's inconsistent with the regulations, either you can't generate scrambles, or you are effectively following TNoodle's definition rather then the regulations' one.

If TNoodle doesn't follow the regulations, then that is a BUG, and it needs to be fixed. I have put a lot of effort into writing automated tests for TNoodle, and Lucas worked to get us running on travis ci: https://travis-ci.org/cubing/tnoodle. We try hard to make sure things work, and follow the regulations. That said, I would never claim that TNoodle is bug free. In my mind, there's no way around the problem you're referring to. It's simply a fact that software does what the code says, not what humans want it to =). I believe that adding more official scramblers would only make this worse.

Here's the thing. From my perspective, most of the changes in the actual scrambling algorithms that I've seen in the last months (and I may have missed some) resulted in an ending product that was either worse, or different without being better. Our scrambling methods were good enough a year ago.

You've brought up some good points here. (Putting on my official hat here) As the leader of the WCA Scrambler Team, I do not believe it is our job to push back when the WRC/board asks us to do something we might not agree with. These guys deal with enough **** as is (don't get me wrong, I do strongly believe that community feedback is critical to the continued success of the WCA), they don't need more of it from us (the scrambler team). They need to know whether what they want is possible, and how long it might take to do. Our opinions don't matter here.

As for changes not having to do with scrambling algorithms, I don't see why they are required for regulation compliance, as opposed to being stuff you guys wanted to do to be helpful. I'm sorry that you spent your free time this way instead of being with your families, but we didn't ask or demand for you all to do it.

If you're referring to changes to TNoodle that aren't in direct response to a regulations change, then you must be referring to one of 2 things:

1. Historically, generating scrambles for a competition was a pain in the ass. It's not only TNoodle's job to follow the regulations, it is also TNoodle's job to make it easier for organizers to run competitions (by providing a nice UI for generating scrambles, and giving people (password protected) pdfs/zips/json, and by doing some crazy **** with a built in webserver to make this all just work on any desktop os anywhere). There are lots of changes and improvements there that do not reflect the regulations at all. https://github.com/cubing/tnoodle/issues/159 is a great example of something we might do for 2014 that is completely unrelated to the WCA regulations.

2. If you were referring to garbage like https://github.com/cubing/tnoodle/tree/master/jsracer, https://github.com/cubing/tnoodle/tree/master/quercus, https://github.com/cubing/tnoodle/tree/master/timer, https://github.com/cubing/tnoodle/tree/master/cubecomps, or any of a number of other things, then I accept full blame for that mess, and I'm sorry for the cruft. TNoodle has always suffered the growing pains of being a personal project of mine that just turned into the official scrambler. Since forever, Lucas has been annoyed with me for not fixing this up, and Tim just reinforced those reminders. https://github.com/cubing/tnoodle/issues/160 will be a high priority item for 2014.

Perhaps the best thing to do would be to take https://github.com/cubing/tnoodle/issues/160 even further, and separate all the "nice helpful" things TNoodle does for organizers away from the bare bones scrambling code? That way, people who care about the regulations could watch this new repository, and they could legitimately complain whenever non bugfix changes occur without a clear change in the regulations. We could call this new project something super straightforward like "wca scrambler" (I know you were never wild about calling the wca scrambler something like TNoodle), and then TNoodle would become a useful tool/wrapper for the wca scrambler project.