Oh yeah, I should have been more clear. What I meant is that going through old logs is useless unless you recorded a specific test session prior to the perceived change and can run that exact same test under the exact same conditions as before. Since no one just creates test cases and runs them for posterity just in case something changes later, old logs won't work. In essence there are too many uncontrolled variables.
What we're left with is the current RNG, which can be tested for standard dev and proper RNG distribution and streaky-ness. So a little bit of work and we can see if the RNG is truly R or at least acceptably so.
But we'll never be able to tell if charm is breaking more than before unless we look at the charm code.
|