Miscellaneous: paladin /sk defense skill bug-230

[Raev]

Originally Posted by koros [You must be logged in to view images. Log in or Register.]

Huh? Unless the effect of defense is astonishingly small, establishing a 95% confidence interval would definitely not take thousands of hits. Also a paired t-test or regression could make it even easier.

Well, you can establish a CI with any number of samples. The problem is shrinking it. I've been gradually putzing around there, and you can see that even with some hundreds of samples the CIs are quite large, assuming I implemented the Wilson Interval correctly.

A glimmering drake [] vs Loin [Various]
Samples: 437
Cooldown: 1.326
Defended: 12.4 - 16.7 (90%, average 14.4, 63 / 437)
Hit Rate: 48.6 - 54.8 (90%, average 51.7, 226 / 437)
Damage: 180 - 380
Per Hit: 233.9
Per Swing: 121.0
Min Hit: 18.0 - 24.9 (90%, average 21.2, 48 / 226)
Interval: 64.0 - 72.0 (90%, average 68.1, 154 / 226)
Max Hit: 8.3 - 13.5 (90%, average 10.6, 24 / 226)

A glimmering drake [] vs Sakuragi [Various]
Samples: 1410
Cooldown: 1.306
Defended: 14.2 - 16.7 (90%, average 15.4, 217 / 1410)
Hit Rate: 34.7 - 38.0 (90%, average 36.3, 512 / 1410)
Damage: 180 - 396
Per Hit: 257.7
Per Swing: 93.6
Min Hit: 22.4 - 27.3 (90%, average 24.8, 127 / 512)
Interval: 70.5 - 75.5 (90%, average 73.0, 374 / 512)
Max Hit: 1.5 - 3.1 (90%, average 2.1, 11 / 512)

Loin is in the early stages of gearing up (Champion's and not much in his other slots), so Sakuragi is probably 100-150 AC (displayed) ahead, yet even with 2000 total samples the 90% CIs (not even the 95% CIs) already overlap for min hits for example.