![]() |
|
#11
|
|||||
|
Quote:
An explicit mental model of the world, with all assumptions stated clearly. You need to be able to articulate your understanding of the world before you can know whether your results indicate an improvement of that understanding. An explicit hypothesis. You need to know what you are looking for in your experiment. Ad-hoc exploratory data gathering can be useful in trying to formulate a hypthothesis, but that exploratory data will not be useful in determining whether a hypothesis is confirmed or rejected. An experiment design. You need to know ahead of time what data you wish to gather, how to gather it, and when to stop the experiment. An example of a flawed experiment would be trying to show that a certain gear combo causes a certain DPS increase, and then stopping your parse as soon as you show that DPS increase. Sanity-checking the resulting data to confirm your assumptions have been met. If not, then your understanding of the world is flawed and your data unusable. You need to first run a different experiment to find and fix the flaws in your assumptions. Run a well-defined, repeatable analysis. You need to know ahead of time what metrics you wish to calculate. You should also do some sort of calculation of statistical confidence, whether frequentist or Bayesian. Scientific integrity. You need to publish your results whether or not they support your hypothesis. If the results violate some of your assumptions, you cannot rely on the results of any data analysis. Quote:
| ||||
|
|