Using electronic health records and Internet search information for accurate influenza forecasting

BMC Infectious Diseases

Table 1 ARGO performance compared to alternative methods for the time period of July 6, 2013 to February 21, 2015

	real-time	forecast 1 week	forecast 2 week	forecast 3 week
RMSE
ARGO	0.315	0.435	0.487	0.459
Ref. [11]	0.469	0.544	0.590	0.578
ar4	0.944	0.954	0.935	0.902
naive	1 (0.374)	1 (0.613)	1 (0.756)	1 (0.869)
MAE
ARGO	0.403	0.446	0.456	0.426
Ref. [11]	0.497	0.614	0.603	0.593
ar4	0.895	0.880	0.872	0.867
naive	1 (0.221)	1 (0.363)	1 (0.480)	1 (0.575)
RMSPE
ARGO	0.449	0.474	0.504	0.461
Ref. [11]	0.655	0.677	0.657	0.691
ar4	1.001	1.018	1.032	1.044
naive	1 (0.126)	1 (0.194)	1 (0.246)	1 (0.293)
MAPE
ARGO	0.481	0.458	0.454	0.419
Ref. [11]	0.625	0.704	0.662	0.676
ar4	0.956	0.965	0.977	0.988
naive	1 (0.101)	1 (0.156)	1 (0.205)	1 (0.251)
Correlation
ARGO	0.995	0.976	0.952	0.942
Ref. [11]	0.989	0.960	0.928	0.904
ar4	0.954	0.871	0.804	0.748
naive	0.951	0.867	0.796	0.727
Error reduction of ARGO over the best available alternative (in %)
RMSE	32.90 [16.38,55.54]	20.07 [5.13,31.38]	17.40 [1.29,28.82]	20.53 [11.82,27.33]
MAE	18.79 [0.23,36.67]	27.44 [10.28,39.18]	24.41 [7.66,34.53]	28.13 [15.84,36.38]
RMSPE	31.50 [21.63,40.84]	29.90 [9.42,41.95]	23.26 [4.69,33.00]	33.32 [19.94,41.69]
MAPE	22.92 [7.93,35.94]	34.95 [18.59,46.76]	31.42 [12.90,43.04]	38.02 [26.00,47.26]

The evaluation metrics between the prediction \( \widehat{p_t} \) and the target \( \widehat{p_t} \) include RMSE \( \left(=\sqrt{\frac{1}{T}\sum_{t=1}^T{\left(\widehat{p_t}-{p}_t\right)}^2}\right),\mathrm{MAE}\left(=\frac{1}{T}\sum_{t=1}^T|\widehat{p_t}-{p}_t|\right),\mathrm{RMSPE}\left(=\sqrt{\frac{1}{T}\sum_{t=1}^T{\left(\frac{\widehat{p_t}-{p}_t}{p_t}\right)}^2}\right),\mathrm{MAPE}\left(=\frac{1}{T}\sum_{t=1}^T\frac{\mid \widehat{p_t}-{p}_t\mid }{p_t}\right) \), and Pearson correlation. The benchmark models include the ensemble method by Santillana et al. [11], an autoregression model with 4 lags, and a naive model, which uses prior week’s ILI level as the prediction for the current week as well as the next 3 weeks. Boldface highlights the best method for each metric in each forecasting time horizon. RMSE, MAE, RMSPE, MAPE are relative to the error of the naive method, i.e., the numbers are the ratio of the error of a given method over that of the naive method; the absolute error of the naive method is given in the round bracket. Table S3 in the Additional file 1 gives the absolute error of all methods. For each forecasting time horizon and each evaluation metrics, the error reduction of ARGO over the best alternative method is given in the second half of the table, together with 95% confidence intervals (in the square bracket) constructed using stationary bootstrap [33] with mean block size of 52 weeks.

ISSN: 1471-2334