Level 55+ for scout roll, yay or nay?

[zodium]

Originally Posted by remen [You must be logged in to view images. Log in or Register.]

Ugh, I knew there would be some dumb response about how "only" 130 people voted in the poll and "only" 70 voted to change.

What would be a pretty bad mistake is to have no idea what a normal sample size in polling is, and try to draw your own conclusions without a basic knowledge of statistics. Using your estimate of 133 people being 5% of the server, that is actually a big enough sample to draw meaningful conclusions. We can say with 95% confidence that somewhere between 44%-60% (52% +/- 8%) of all the players on the server are in favor of a change based on the results of this poll. Facts are fun!

Fun fact: Drawing a sound inferential conclusion is an exercise in mind boggling difficulty. Like, you just won't believe how hard it is! Wow! It is, certainly, far too hard to bother going through for a P99 poll. No one qualified to do it would even try.

Now, savagely criticizing a gross misapplication of statistics? That is a different story. Here are a few minor problems with your attempt:

1. As even a first year statistics student would know, that is an on-its-face incorrect interpretation of your computed result. It's such a textbook mistake, the answer given appears as a standard lure in every first-year MCT question bank for statistics exams. Without, at least, a technically correct interpretation, we can never hope for our conclusion to be sound. Grade: C.
2. Your confidence interval, as computed, contains both possible outcomes. This implies the sample size is, in fact, too low to soundly infer a conclusion. Considering your tone, this is at best a very embarrassing error. Grade: F.
3. This is not something anyone who isn't like a statistician or methodologist are supposed to know, I guess, but bears stating to put into perspective how difficult sound inference is: a sound frequentist interpretation can't ever be inferred from this poll, because a stopping rule was not specified in advance. Any conclusion may theoretically be obtained, simply by arbitrarily choosing when to stop collecting data. Applying a Bayesian credible interval technique instead may technically resolve this, but at current sample sizes and proportions is uh, extremely unlikely to produce a result that is both sound and unequivocal in support of either possible outcome. Also, post-hoc/"fishing expedition" analyses ipso facto imply dramatically reduced validity.

Just take the poll on its face, friends. Don't grasp at statistical straws. As the old saying goes,

Originally Posted by Ronald Fisher

To consult the statistician after an experiment is finished is often merely to ask him to conduct a post mortem examination. He can perhaps say what the experiment died of.

In conclusion: Slay monsters, not statistics! [You must be logged in to view images. Log in or Register.]