[Baugi]

Originally Posted by Rygar [You must be logged in to view images. Log in or Register.]

Not sure what exactly I'm looking at, but compiling and graphing data based on log files is generating a healthy afternoon erection. Please continue

If the channeling code is correct, it should accurately predict actual channeling outcomes. That is, you plug in the variables (channel skill, player level, spell level diff) and it gives you the probability of a successful cast.

I walked through the log files I had to find all the relevant casts along with the relevant variables (event file). If you have a collection of casts, the expected number of successes is just the sum of the individual probabilities.

In the graph, for each model(ie azxten's code vs EQEmu code), I'm just calculating the probability of success for each cast according to the model and grouping them together based on their probability of success, then plotting the expected ratio of success in the group against the actual ratio of success for the group.

Ideally, a good predictor should follow the diagonal line because that represents the expected success ratio matching the actual success ratio. The azxten code more or less follows the slope which is a good sign that the variables and scaling are correct, but does bounce around a lot even given the sample size. Also, it consistently overpredicts the probability for groups with more events which makes it a bit worse than is obvious on the graph (possibly explained by the movement interrupt issue).

The EQemu code in contrast is just bad, you can see it curled up in the corner which means it consistently overpredicts and the scaling is likely wrong. The contrast is bad enough that it's not worth trying to compare them in a more sophisticated way. I did play a little with including the push interrupt term and assuming push increased linearly with number of hits, but the scaling wasn't any good.

If someone is bored, they could try fitting a better function with the event data and cross-validation. Better might be to test if the historical behavior matches up well with live behavior. If so, you could do much more controlled tests and get rid of the movement interrupt issue. I'm also under the impression that P99's interrupt code is different from the stock EQEmu code so it should really be compared as well, but to be honest I don't care to try and extrapolate out the behavior when the devs have access to the source.

My scripts are just in the repo btw if that's easier than reading the wall of text - scripts