Experiment Results

[BenChristman1]

My experiment is done, and I have the results. I was testing 2 main things. The first was which method gets you the best single time relative to their average time. The second thing was which method performs best in a competition relative to their global average. Keep in mind that only 36 people responded to the survey. Like any other experiment, the results will be more accurate the larger that your sample size gets.

Spoiler: Which method gives the best single?

From the test, I concluded that Petrus gives you the best singles relative to your averages. In the survey, Petrus users' singles were (on average) 60% of their global averages.

Spoiler: Which method gives the best comp averages?

From the test, I concluded that Hawaiian Kociemba gives the best comp averages. Only 1 person who responded to the survey used this method, though. Their comp average is about 108% of their global average. The second-place method was Roux. In this survey, Roux users' comp averages were the second-lowest with about 110%.

Spoiler: Other random stuff I tested

Spoiler: Average Times

4 people average less than 10 seconds!

Spoiler: Single Times

The best single time was 5.27 seconds!

Spoiler: Main Cube

Most people use Moyu cubes!

Spoiler: Method

Most people use CFOP!

Spoiler: Have you been to a WCA Competition?

Most people have been to a WCA competition!

Thank you to everybody who responded to the survey! Let me know if you have any questions!

 

[ProStar]

I'd like to point out that methods don't have any correlation to comp times, it comes down purely to the person. If the people who used CFOP happen to have gotten less nervous, then it doesn't mean CFOP users do better in comps. Also, since methods like Hawiian Kociemba and like Waterman have almost no users overall, the percentages for method use can be skewed, as in reality the percentage that use Hawiian Kociemba are almost 0(the same thing applies to other methods). Still a cool survey with interesting results. Also GTS3 FTW!

 

[BenChristman1]

I'd like to point out that methods don't have any correlation to comp times, it comes down purely to the person. If the people who used CFOP happen to have gotten less nervous, then it doesn't mean CFOP users do better in comps.

Going into the experiment, I thought that it might have a slight correlation. For example, Roux users might have more DNFs because they have M moves at the end of there solves, which will make their averages worse. Obviously, that wasn't the case though.

Also, since methods like Hawiian Kociemba and like Waterman have almost no users overall, the percentages for method use can be skewed, as in reality the percentage that use Hawiian Kociemba are almost 0(the same thing applies to other methods).

Keep in mind that only 36 people responded to the survey. Like any other experiment, the results will be more accurate the larger that your sample size gets.

Still a cool survey with interesting results.

Thanks!

 

[brododragon]

I'd like to point out that methods don't have any correlation to comp times, it comes down purely to the person. If the people who used CFOP happen to have gotten less nervous, then it doesn't mean CFOP users do better in comps. Also, since methods like Hawiian Kociemba and like Waterman have almost no users overall, the percentages for method use can be skewed, as in reality the percentage that use Hawiian Kociemba are almost 0(the same thing applies to other methods). Still a cool survey with interesting results. Also GTS3 FTW!

I think methods can have correlations to averages. Let me explain: Day there are two methods: A and B. after adding all the chances of getting lucky (in one way or another), you get 1/5 chance to get lucky with method A and 1/20 for method B. In method B, you're probably not going to get a lucky solve in an Ao5, but with A, you are. Lucky solves mean you get your best non-luck (or less lucky) solve counted. This same idea can be applied to things you might not even consider "lucky". Even if your method doesn't have that good look-ahead normally, if it's got a high chance to have good look ahead you will have very good look ahead more often.

 

[ProStar]

I think methods can have correlations to averages. Let me explain: Day there are two methods: A and B. after adding all the chances of getting lucky (in one way or another), you get 1/5 chance to get lucky with method A and 1/20 for method B. In method B, you're probably not going to get a lucky solve in an Ao5, but with A, you are. Lucky solves mean you get your best non-luck (or less lucky) solve counted. This same idea can be applied to things you might not even consider "lucky". Even if your method doesn't have that good look-ahead normally, if it's got a high chance to have good look ahead you will have very good look ahead more often.

I'm saying that methods won't be better at competitions than another. Like, the luckiness doesn't change in comp versus at home, and a method isn't better at a competition than another

 

[Kit Clement]

Unless you randomly assigned the methods that the participants used, this is by no means an experimental design and you can't attribute cause/effect to method and single/average ratio.

Looking through the data, it looks like you also had 2 Petrus users (unless somehow you had more than one person list their method as hawaiian kociemba), meaning that the data you're showing aren't likely showing any real significant differences, and are just a product of the sample you collected rather than being indicative of a global effect.

 

[BenChristman1]

Unless you randomly assigned the methods that the participants used, this is by no means an experimental design and you can't attribute cause/effect to method and single/average ratio.

Looking through the data, it looks like you also had 2 Petrus users (unless somehow you had more than one person list their method as hawaiian kociemba), meaning that the data you're showing aren't likely showing any real significant differences, and are just a product of the sample you collected rather than being indicative of a global effect.

Keep in mind that only 36 people responded to the survey. Like any other experiment, the results will be more accurate the larger that your sample size gets.

@Kit Clement did you respond to the survey?

 

[Kit Clement]

@Kit Clement did you respond to the survey?

No, I didn't notice the thread earlier.

I saw that note you made, I was specifically pointing out the group sizes are way too small (many are 1-2 people) to draw any conclusions, as well as the fact that you can't make the cause/effect conclusions you made regardless of sample size.

 

[BenChristman1]

I saw that note you made, I was specifically pointing out the group sizes are way too small (many are 1-2 people) to draw any conclusions, as well as the fact that you can't make the cause/effect conclusions you made regardless of sample size.

I also originally thought that I would get more people to respond, but clearly, that wasn't the case. At least CFOP is somewhat accurate.

 

[ProStar]

I also originally thought that I would get more people to respond, but clearly, that wasn't the case. At least CFOP is somewhat accurate.

Not necessarily. In reality, CFOP has many more users over Hawiian Kociemba or Petrus or ZZ or Waterman than is shown

 

[BenChristman1]

Not necessarily. In reality, CFOP has many more users over Hawiian Kociemba or Petrus or ZZ or Waterman than is shown

The number of people who use other methods doesn't affect how people who use CFOP perform.

 

[Etotheipi]

Dang @Cubingcubecuber screwing with the stats with his Hawiian Kociemba.