Development of a skill assessment tool for the Korea operational oceanographic system

Abstract

A standard skill assessment (SA) tool was developed and implemented to evaluate the performance of operational forecast models in the Korea operational oceanographic system. The SA tool provided a robust way to assess model skill in the system by comparing predictions and observations, and involved the computation of multiple skill metrics including correlation and error skills. User- and system-based acceptance criteria of skill metrics were applied to determine whether predictions were acceptable for the system. To achieve this, the tool produced a time series comparison plot, a skill score table, and an advanced summarized diagram to effectively demonstrate the multiple skill scores. Moreover, the SA was conducted to evaluate both atmospheric and hydrodynamic forecast variables. For the atmospheric variables, acceptable error criteria were preferable to acceptable correlation criteria over short timescales, since the mean square error overwhelmed the observation variance. Conversely, for the hydrodynamic variables, acceptable root mean square percentage error (e.g., pe (rms)) criteria were preferable to acceptable error (e.g., e (rms)) criteria owing to the spatially variable tidal intensity around the Korean Peninsula. Furthermore, the SA indicated that predetermined acceptance error criteria were appropriate to satisfy a target central frequency (f (c) ) for which errors fell within the specified limits (i.e., the f (c) equals 70%).