Hongren Shen, Bryan A. Tolson, Juliane Mai (2022). Time to Update the Split Sample Approach to Hydrological Model Calibration. Proceedings of the GWF Annual Open Science Meeting, May 16-18, 2022.

Model calibration and validation are critical in hydrological model robustness assessment. Unfortunately, the commonly used split-sample test (SST) framework for data splitting requires modelers to make subjective decisions without clear guidelines. Unlike most SST studies that use two sub-periods (i.e., calibration and validation) to build models, this study incorporates an independent model testing period in addition to calibration and validation periods. Two hydrological models are calibrated and tested in 463 CAMELS catchments across the United States using 50 different data splitting schemes. These schemes are established regarding the data availability, length, and data recentness of the continuous calibration sub-periods (CSPs). A full-period CSP is also included in the experiment, which skips model validation entirely. The results are synthesized regarding the large sample of catchments and are comparatively assessed in multiple novel ways, including how model building decisions are framed as a decision tree problem and viewing the model validation process as a formal testing period classification problem, aiming to accurately predict model success/failure in the testing period. Results span different climate and catchments make conclusions generalizable. Strong patterns show that calibrating models to older data and then validating models on newer data produces inferior model testing period performance and should hence be avoided. Calibrating to the full available data and skipping model validation entirely is the most robust split-sample decision. Results strongly support revising the traditional split-sample approach in hydrological modeling.

AOSM2022 IMPC First Author: Hongren Shen, University of Waterloo Additional Authors: Bryan A. Tolson, University of Waterloo; Juliane Mai, University of Waterloo