Table 1 Overview of studies on forecasting patient arrivals in emergency departments (n = 33)
Author (Year) / Country | EDs analyzed / Time frame | Forecast object, period and horizon | Predictors tested | Forecasting models | Most frequently retained variables | Partitioning of the dataset for training, test and validation | Performance metrics | Main Results |
|---|---|---|---|---|---|---|---|---|
Chen et al. [17] (2011) / Taiwan | ED of a regional teaching hospital with intensive care beds in Kaohsiung City/ 01/2005 – 09/2009 | Number of monthly general, traumatology, and pediatric arrivals over a 9-month horizon | - | ARIMA | - | Training: 01/2005 −12/2008. Test: 01/2009–09/2009 | MAPE | General patients: ARIMA model with MAPEs in the interval (13.71%—41.61%) and average 19.59% Traumatology patients: ARIMA model with MAPEs in the interval (10.16%—19.12%) and average 12.39% Pediatric patients: ARIMA model with MAPEs in the interval (5.73%—54.24%) and average 29.08% |
Boyle et al. [22] (2012)/ Austrália | EDs of 27 public hospitals mixed urban e regional in Queensland / 2005–2009 | Monthly arrivals over a 12-month horizon | Calendar: day of week, month of year, holidays, and days before and after holidays | ARIMA, MLR and ES | All variables in the model MLR | Training: 01/2005- 12/2008. Test: 01/2009–12/2009 | MAPE | For all 27 hospitals: MAPEs in the interval (7%—25%) and average 12.30% |
Marcilio et al. [42] (2013) / Brazil | High complexity ED in the southern region of the city of São Paulo, operating 24/7 / 01/2008–12/2010 | Daily and monthly arrivals over 7 and 30-day horizons | Calendar: day of week, holidays, and days before and after a holidayClimate: daily average temperature | GLM, GEE and SARIMA | All variables in the best performing models (GLM and GEE) | Training: 01/2008–09/2010 Test: 10/2010–12/2010. After forecasting 7 and 30 days in October, the values observed of October were incorporated into the training set and the models re-estimated to November forecast. This process was repeated in the month of December | MAPE | GLM and GEE with MAPEs of 4.5–9.9% over 7-day horizon, with and without temperature as predictor. GLM and GEE with MAPEs ranging of 8.7–12.8% over 30-day horizon, with and without temperature as predictor |
Menke et al. [43] (2014) / Country not reported | ED of a tertiary hospital, location not disclosed / 02/2007–12/2009 | Daily arrivals. Horizon not reported | Calendar: day of week and special events Climate: weather and air quality | MLP with back propagation | All variables to estimate the MLP algorithm | There was no division between training and testing | R2 | The adjusted R2 is 0.957. 95% of the time, the MLP estimated an error within 20 arrivals |
Kadri et al. [44] (2014) / France | Paediatric Emergency Department of the Lille Regional Hospital Centre, operating 24/7 / 01/2012–12/2012 | Total number of daily arrivals, unplanned arrivals (G2) and unexpected arrivals (G4) over 1 to 7-day horizons | - | ARMA | - | Training: 01/2012–12/2012. Test: 12/2012–12/2012 | ME | Total arrivals: ME of 3.79–9.03. Unplanned arrivals (G2): ME ranged of 5.5–9.26. Unexpected arrivals (G4): ME of 31–72, over all horizons |
Bergs et al. [45] (2014)/ Belgium | Four EDs of a university hospital and three regional hospitals in the Flemish region / 01/2005–12/2011 | Monthly arrivals over a 12-month horizon | - | ETS | - | Training: 01/2005–12/2010 Test: 01/2011–12/2011 | MAPE, MASE and MAE | MAPEs ranged of 2.63–4.76% and MASE of 0.53–0.68 in forecasting the four EDs |
Calegari et al. [46] (2016) /Brazil | ED of a public, tertiary teaching hospital in the city of Porto Alegre, serving high complexity patients, operating 24/7 / 01/2013–05/2015 | Total number of daily arrivals, urgent arrivals and very urgent arrivals over horizons of 1, 7, 14, 21 and 30 days | Calendar: month of year and day of week. Climate: minimum, maximum and average temperature, amount of rain, wind speed, relative humidity and hours of insolation | SS, HW, SARIMA and MSARIMA | All calendar and climate variables in the MSARIMA models | Training: 01/2013–03/2015 Test 30-days: (i) 03/2015–04/ 2015, (ii) 01/2015–02/2015 and (iii) 05/2015–05/2015. After forecasting of (i). The values observed of (i) were incorporated into the training set and the models re-estimated to forecast of (ii). This process was repeated on test set (iii) | MAPE | Total arrivals of all horizons: SS with MAPEs of 2.91%−11.35%. Urgent and very urgent arrivals of all horizons: SARIMA obtained MAPEs ranging of 3.98%−15.71% and 7.19%−17.23% respectively |
Xu (2016) [18] /China | EDs from two hospitals in DaLian, LiaoNing Province. ED-A is large while ED-B is small / 01/2012–12/2013 | Daily arrivals over 1 and 7-day horizons | Calendar: day of week, month of year, public holidays and school holidays, day after a holiday Climate: maximum and minimum daily temperature | ARIMA, ARIMAX, ARIMA-LR, ARIMA-LR with smoothing, ARIMA- ANN and GLM | Group 1: temperature variables, group 2: holiday variables, and group 3: calendar variables. In ED-A, the three groups of variables were retained by the GLM, ARIMAX, ARIMA-LR and ARIMA-ANN models. In ED-B, group 2 was retained in all models | Training: 01/2012–06/2013 (547-days). Test: 07/2013–12/2013 (184-days) | MAPE and RMSE | ED-A and ED-B: ARIMA-LR obtained values of MAPEs ranging of 5.8%−13.1% and RMSEs of 5.37–136.4 over 1-day horizon. ED-A and ED-B: ARIMA-LR (smoothing) obtained MAPEs ranging of 6.1%−12.9% and RMSEs of 5.33–147 over 7-day horizon |
Juang et al. (2017) [19] /Taiwan | ED of a medical center in southern Taiwan / 01/2009–12/2016 | Monthly arrivals within a 12-month horizon | - | ARIMA | - | Training: 01/2009–12/2015 Test: 01/2016–12/2016 | MAPE | ARIMA (0, 0, 1), obtained a MAPE of 8.91% |
Hertzum (2017) [47] /Denmark | Four EDs of mid-sized hospitals in the Zealand region / 01/2012–01/2015 | Hourly arrivals over 1, 2, 4, 8 and 24-h horizons | Calendar: time of day, day of week and month of year | MLR, ARIMA and Naive | The months of year, days of week, and times of day in MLR | Training: 01/2012–12/2014 Test: 01/2015–01/2015 | MAPE, MASE and MAE | 1-h prediction in the 4 EDs: ARIMA e MLR obtiveram MAPEs variando entre 47%−58% e MASE de 0.72–0.77. 2, 4, 8, and 24-h predictions for ED2: ARIMA and MLR with MAPEs of 41% and 37% (2-h), 34% and 34% (4-h), 21% and 26% (8-h), and 11% and 9.9% (24-h) |
Carvalho-Silva et al. [23] (2018) /Portugal | ED of a public hospital in the Minho region / 01/2012–12/2014 | Weekly and monthly arrivals over one-week, three-week, one-month and twelve-month horizons | - | ARIMA, HW, Multiplicative HW, Moving average and ES | - | Training: 01/2012–12/2013 Test: 01/2014–12/2014 and three weeks of 01/2014 | MAPE | ARIMA: MAPEs of 5.22% and 6.34%, in weekly and monthly forecasts |
Yucesan (2018) [4] / Turkey | ED of a private hospital in Trabzon / 05/2015–04/2017 | Daily arrivals. Horizon not reported | Calendar: day of week, day of year and month of year | MLR, ARIMA, ANN, ES, ARIMA-ANN and ARIMA-LR | MLR and ANN models retained all calendar variables. Best performing hybrid methods did not retain variables | Training: 2016 (365-days). Didn't have the test set | MAPE | ARIMA-ANN and ARIMA-LR: MAPEs of 0.49% and 0.92% respectively |
Asheim et al. [48] (2019) /Noruega | ED of St. Olav’s University Hospital, Trondheim, operating 24/7 / 01/2010–12/2018 | Hourly arrivals over 1, 2 and 3-h horizons | - | Poisson regression | - | Training: 01/2010–12/2016 Validation: 01/2017–12/2017 Test: 1, 2 and 3-h | MAPE | 1, 2, and 3-h forecasts: MAPEs were 57%, 43% and 36%, respectively |
Jilani et al. [15] (2019) / United Kingdom | Four EDs in the UK, locations not disclosed / 01/2011–12/2015 | Weekly and monthly arrivals over 4-week and 4-month horizons | - | Fuzzy Time Series (FTS), ARIMA and NN | - | Training: 01/2011–12/2015 (260 datapoints). Test: 4-weeks and 4-months | MAPE and RMSE | For all EDs: MAPEs ranged from 2.6%−4.7% using FTS in weekly forecasts and MAPEs from 2.01%−2.81% and RMSE from 57.30–167.89 in monthly forecasts |
Whitt et al. [3] (2019) /Israel | ED of Rambam Hospital, Haifa, operating 24/7 / 01/2004–10/2007 | Daily arrivals over 1 to 7-day horizons | Calendar: days of week, month of year and holidays. Climate: precipitation, maximum and minimum daily temperature | MLR, SARIMA, SARIMAX and MLP | The days of week, the month of year, holidays and temperatures in MLR, SARIMAX and MLP | tenfold cross validation in all datasets for MLP | MAPE and MSE | SARIMAX: 8.4% MAPE in 1-day forecast. MSE between 193–211, in forecasts of 1 to 7-days |
Khaldi et al. [27] (2019) /Morocco | ED in the city of Fes, operating 24/7 / 2010–2016 | Weekly arrivals over one-week week horizon | - | ANN, ARIMA EEMD-ANN and DWT-ANN | - | ARIMA used 80%/20% train/test partitions. Remaining models used training: 70%, Validation: 15%, Test: 15% | RMSE, MAE and R2 | EEMD-ANN had the best performance: RMSE of 52.86 and MAE of 39.88 |
Zhang et al. [35] (2019) /China | Radiology ED of a large hospital in Sichuan / 01/2013–12/2016 | Daily arrivals over a 1-day horizon | Calendar: temporal factors related to annual, quarterly, monthly and weekly periodicities and the effect of holidays | ARIMA, SVR and ARIMA-SVR | Temporal factors annual, quarterly, monthly, weekly and the effect of holidays | Training: 80% Test: 20% tenfold cross validation only in training set to determine the hyperparameters of the SVR and ARIMA-SVR methods | MAPE, RMSE, MAE and relative error (RE) | ARIMA-SVR achieved the best performance: MAPE of 7.02% and RMSE of 19.20 |
Choudhury and Urena [1] (2020) / USA | ED of a hospital in Des Moines, Iowa / 2014–2017 | Hourly arrivals over a 30-day horizon | - | TBATS, HW, NN and ARIMA | - | Training: 01/2014–07/2017 Test: 08/2017 | RMSE and ME | ARIMA had the best performance: RMSE of 1.55 and ME of 1.00 |
Yousefi et al. [21] (2020) /Brazil | ED of a hospital in Belo Horizonte, operating 24/7 / 01/2014–11/2016 | Daily arrivals over 1 to 7-day horizons | Calendar: soccer match events, weekends, holidays, day before and after holidays | LSTM | Soccer match events, weekends, holidays | Training: 70% Test: 30% | MAPE and R2 | MAPE values from 4.89% to 6.31% (average 5.55%) and R2 values from 0.86 to 0.99 (average 0.940) over all horizons |
Harrou et al. [16] (2020) / France | Paediatric ED of the CHRU-Lille Hospital, operating 24/7 / 01/2011–11/2013 | Hourly and daily arrivals over a 1 to 4-h and 1-day horizons | - | RNN, LSTM, BiLSTM, ConvLSTM, RBM, CNN, GRU and VAE | - | Training: 70% Test: 30% | RMSE, MAE, R2 and EV | VAE was the best performing model, with RMSE values from 0.41 to 2.74 and MAE values from 0.30 to 2.32 over all horizons |
Rocha and Rodrigues [5] (2021) /Portugal | ED of a Portuguese hospital, location not disclosed / 01/2009–12/2018 | Daily and hourly arrivals over 4 to 24-h horizons | Calendar: year, month of year, day of week, time of day and holidays | Snaive, ESD, SARIMA, AR-NN, RNN, XGBoost, RNN-1L, RNN-3L, RNN-1L–XGBoost and Ensemble model | AR-NN, RNN, XGBoost, RNN-1L, RNN-3L, RNN-1L–XGBoost and Ensemble models retained all variables | Training: 01/2009–12/2016 Validation: 01/2017–12/2017 Test: 01/2018–12/2018 | RMSE, sMAPE and MAE | RNN-1L: RMSE ranging of 4.8–26 and sMAPE of 4.3%−21.3%; RNN-3L: RMSE ranging of 4.7–26.1 and sMAPE of 4.2%−23.1%; and RNN-1L–XGBoost: RMSE ranging of 4.7–28.4 and sMAPE of 4.7%−23.2% over all horizons |
Vollmer et al. [20] (2021) / England | EDs at St Mary's Hospital and Charing Cross Hospital in London, both operating 24/7 / 01/2011–12/2018 | Daily arrivals over 1, 3 and 7-day horizons | Calendar: day of week, month of year, public holidays, and school holidays. Climate: precipitation, maximum and minimum temperature on previous day | ARIMA, ETS, STLM, StructTS, GLMNET, RF, GBM and KNN | ML algorithms (GLMNET, RF, GBM and KNN) retained all variables | sixfold time series cross validation in all datasets | MAPE and MAE | St Mary’s hospital: MAPEs of 6.9%−8.3% using time series models and MAPEs of 6.8%−7.4% using ML algorithms. Charing Cross Hospital: MAPEs ranging of 8.5%−12% using time series models and MAPEs from 8.6%−10.1% using ML algorithms |
Erkamp et al. [6] (2021) / Netherlands | ED of Jeroen Bosch Hospital in the Hertogenbosch, operating 24/7 / 06/2016–12/2019 | Daily arrivals over a horizon not reported | Calendar: day of week, month of year, summer vacation, school holidays and public holidays Climate: wind speed, minimum, average and maximum temperatures, radiation, pressure, visibility, cloudiness, humidity and precipitation | MLR | Calendar variables, temperature maximum, radiation, pressure, visibility and humidity | Training: 06/2016–12/2018 Test: 01/2019–12/2019 | MAPE | MAPEs of 8.71%, retaining only calendar predictors and 8.68%, retaining calendar and climate predictors |
Sudarshan et al. (2021) [7] / Denmark | ED of a public teaching hospital in Esbjerg, operating 24/7 / 05/2015–10/2017 and 01/2019–12/2019 | Daily and weekly arrivals over 1, 3, and 7-day horizons | Calendar: time of day, day of week, day of month, month of year, day of year, holidays and school holidays. Climate: temperature, wind speed, wind direction, cloud visibility, cloud cover and dew point | RF, LSTM and CNN | Day of week, day of month, month of year, day of year, holidays, temperature, wind speed, cloud visibility, cloud cover and dew point retained by all algorithms | tenfold cross validation in all datasets | MAPE and MSE | LSTM: average MAPEs of 9.31% and 8.91%, average MSEs of 193.25 and 190.46, on the three and seven-days forecasts. CNN: average MAPEs of 9.24% and 10.69%, average MSEs of 192.84 and 232.39, on the three and seven-days forecasts |
Pekel et al. [39] (2021) /Turkey | ED of a public hospital in Istanbul / 01/2011–12/2012 | Daily arrivals within 148 days horizon | Calendar: month of year, day of week, and holidays. Climate: maximum daily temperature | Bayesian ANN model, GA-ANN and PSO-ANN | ML algorithms (ANN, GA-ANN and PSO-ANN) retained all variables | tenfold cross validation only in training set | MAPE, RMSE, MAE, MSE and R2 | PSO-ANN obtained the lowest values in all performance metrics evaluated, with MAPE of 6% and RMSE of 53.29 |
Harrou et al. [24] (2022) /France | Paediatric ED of the CHRU-Lille Hospital, operating 24/7 / 01/2011–12/2012 | Daily arrivals of the following types: non-urgent, urgent, unexpected, biology, radiology, scanner and echography over a 150-day horizon | - | DBN, RBM, LSTM, GRU, CNN-GRU, CNN-LSTM, GAN-RNN, SVR and RR | - | Training: 80% (first 580-days of each time series) Test: 20% (remaining 150 days). Algorithms’ hyperparameters determined through minimization of the cross-entropy error | MAPE, RMSE, R2, EV and MAE | DBN, RBM and CNN-GRU showed superior performance, with mean values of MAPEs between 4.09%−7.52% and RMSE of 0.63–0.94 |
Petsis [40] (2022) /Greece | ED of a general and public hospital in Ioannina, operating on odd days / 03/2013–12/2019 | Daily arrivals over a 1- and 2-days horizon | Calendar: day of year, month of year, day of month, day of week and week of year. Public and local holidays, school holidays and special events. Climate: daily average, minimum and maximum temperatures, amount of rain per day and average wind speed per day | XGBoost | Algorithm retained all variables | Training: 80% (first 870-days of the time series) Test: 20% (remaining 217-days). Algorithm trained through cross-validation | RMSE, MAPE and MAE | One-Day forecasting: RMSE of 22.96 and MAPE of 6.5%. Two-Day forecasting: RMSE of 23.9 and MAPE of 6.91% |
Zhang et al. [41] (2022) /China | ED of a public hospital in Hefei / 11/2019–11/2020 | Daily and hourly arrivals over a 90-day horizon | Calendar: day of week, time of day, month of year, season of year and holiday Climate: Daily temperature (min, max and mean), mean wind speed, air quality level, precipitation, weather and variables related to the air quality index | MLR, KNN, SVR, Ridge, XGBoost, RF, AdaBoost, Gradient Boosting, Bagging and LSTM | All variables (except month of year) retained by all ML algorithms on hourly arrivals. All variables (except time of day, month of year) retained by all ML algorithms and MLR on daily arrivals | Training: 11/2019–08/2020 Test: 09/2020–11/2020 | RMSE, MAPE and MAE | Daily Arrivals: SVR was superior with RMSE of 26.84 and MAPE of 8.81%. Hourly Arrivals: LSTM and XGBoost were best performers with RMSEs of 4 and 4.5 and MAPEs of 49% and 44%, respectively |
Zhao et al. [49] (2022) /Singapore | ED of the Singapore General Hospital, operating 24/7 / 01/2015–12/2019 | Daily arrivals over horizons of 546, 327, 108 and 53 days | Calendar: day of week Climate: temperature and daily relative humidity | ARIMA, Prophet, CNN, ConvLSTM, BiLSTM, DLSTM, DGRU and DRNN | ML algorithms (CNN, ConvLSTM, BiLSTM, DLSTM, DGRU and DRNN) retained all variables | Training: 70% Test: 30% for 6 months to 5-year-long time series sizes | MAPE, RMSE and MAE | For 1 and 5-year-long time series, DLSTM yielded the best results with MAPE values of 5.67% and 5.72%, and RMSEs values of 25.29 and 24.71, respectively For 6 months and 3-year-long time series, DRNN yielded the best results with MAPEs values of 5.41% e 5.70% |
Gafni-Pappas et al. [50] (2023) / USA | ED of the St. Joseph Mercy Hospital in Michigan, operating 24/7 / 01/2017–12/2019 | Daily arrivals over a 365-day horizon | Calendar: day of week, week of month, month of year Climate: Daily temperature maximum, precipitation, cloud cover, relative humidity, average pressure, daily change pressure, wet bulb, solar radiation, air quality level, ozone concentration, and percent flu visits | ARIMA, Prophet, ETS, GBM, RF, and Prophet-XGBoost | The day of week, week of month, temperature, average pressure, percent flu visits, retained by all ML algorithms | Training: 01/2017–12/2018 Test: 01/2019–12/2019 fivefold time series cross validation only in training set in ML algorithms | RMSE | RF showed superior performance, with mean value of RMSE of 18.94 |
Hu et al. [51] (2023) / USA | ED of a quaternary-care teaching hospital in New York, operating 24/7 / 01/2018–01/2021 | Hourly arrivals over 12-h shifts | Calendar: day of week, day versus night, month of year, season, holidays, Recent arrival count 1 and 7-day prior, 30-day moving average, Climate: temperatures (min, max), precipitation, snow, wind, and hot-weather indicator | LR, SARIMA, SARIMAX, XGBoost, and regression tree (RT) | All variables (except month of year, recent arrival count 1-day prior, temperature max, and hot-weather indicator) in SARIMAX model | Training: 01/2018–01/2019 Test: 02/2019–01/2020 COVID test set: 02/2020–01/2021 tenfold cross validation only in training set in ML algorithms | MAPE and RMSE | SARIMAX showed superior performance, with mean values of MAPE of 8.70% and RMSE of 14.65 for non-COVID test set |
Reboredo et al. [52] (2023) / Spain | ED of a teaching hospital in Santiago, operating 24/7 / 01/2015–12/2020 | Daily arrivals over horizons of 1 and 730-days | Calendar: day of week, weekends, season and past mean values | Poisson regression and INGARCH model | All variables in INGARCH model | Training: 01/2015–12/2018 Test: 01/2019–12/2020 | MAE and MSE | INGARCH showed superior performance, with values of MAE of 24.52 and MSE of 979.06 in forecast of 730-days |
Rostami-Tabar et al. [53] (2023) / UK | ED of a hospital, location not disclosed / 04/2014–02/2019 | Hourly arrivals over 1 to 48-h horizons | Calendar: hour of day, day of week, day of year, week of year, holidays, and 24 h lags of holidays and events sporting Climate: temperature | ADAM, Poisson regression, QR, GAM, GAMLSS, Naive, ETS, TBATS, and Prophet, GBM | All variables in ADAM, Poisson regression, QR, and GBM | Training: 04/2014–02/2018 Test: 03/2018–02/2019 | RMSE, Quantile Bias, and Pinball score | ADAM showed the best performance, with value of RMSE of 0.0896 in forecast of 48-h |