Speaker
Description
Comparing two number sets is a foundational component of the scientific process. We should not assume, however, that scientists know the best approach to conducting an appropriate data-model comparison for their particular need. There are many ways to compare two number sets, though, and it can be a confusing task to choose the right process for a particular situation. For example, certain metrics work well only when the distribution of data-model differences is Gaussian, such as root mean square error, correlation coefficient, and mean error. Other metrics work better when the number set distributions, or the distribution of their differences, is non-Gaussian. In addition, each metric only assesses a specific aspect of the data-model relationship. A complementary set of metrics should be employed to conduct a robust comparison. Depending on the scientist's objective in doing the comparison, some metrics could be far more important than others. This knowledge of the strengths and limitations of each metric, and the right combination of metrics for certain types of studies, is, not widely known and applied in scientific research. How many scientists actually work is that they use the metrics they already know, and often do not consider using other, potentially more appropriate ones. Scientists need guidance about the many metrics available to them. It is advocated that this guidance should be embedded into the analysis software.