Speaker
Description
Skill scores are useful secondary data-model comparison metrics that transform the value of another metric relative to that for a reference model. A well-known skill score for "continuous metrics analysis – those comparison techniques that employ the exact values of observations and model results – is prediction efficiency (PE), based on mean square error as the metric and the average of the observed values as the reference model. There is no equivalent skill score like PE for event detection analysis – those comparison techniques that convert the exact values into yes-no event status and create metrics from the contingency table counts. Here, we present two such options, one based on the proportion correct metric and another based on the critical success index metric. Like PE, these new skill scores use the observations as the reference model, which provides complete independence of the reference model from the accuracy of the new model. It is demonstrated that these skill scores provide context for model evaluation that is unique to other existing event detection metrics and valuable for the assessment, in particular with respect to comparing the new model's performance against an existing model. It is advocated that these new skill scores should be incorporated into space physics analysis software packages.