A Methodological Approach for Selecting Climate Datasets for Drought Prediction Using ROC Analysis and Youden’s Index
DOI:
https://doi.org/10.54386/jam.v28i3.3250Keywords:
Climate datasets, Drought assessment tools, Drought forecasting, ROC analysis, Standardized Precipitation Index (SPI), Youden’s IndexAbstract
This study presents a methodological approach for evaluating and selecting climate datasets for drought prediction in Guyana, based primarily on the combination of Receiver Operating Characteristic (ROC) analysis and Youden’s Index (J). Traditional hydrological metrics—Nash–Sutcliffe Efficiency (NSE) and Kling–Gupta Efficiency Modified (KGEM) were applied only as comparators to highlight their limitations for event detection. Ten climate datasets available in CARiDRO were assessed at two Standardized Precipitation Index (SPI) time scales—3 months (SPI3) and 12 months (SPI12) using records from two contrasting stations: Georgetown (coastal) and Timehri (inland). ROC–Youden rankings were stable across stations and time scales and revealed clear leaders. At Georgetown, the highest J values were obtained by RCP 4.5 for both SPI3 (J = 0.44) and SPI12 (J = 0.48), followed by aexsk (J = 0.22 and 0.34, respectively). At Timehri, aexsk achieved the highest performance for both SPI3 (J = 0.21) and SPI12 (J = 0.35), while RCP 4.5 showed strong but second-best performance only at SPI12 (J = 0.32). In contrast, NSE and KGEM yielded inconsistent rankings between SPI3 and SPI12, underscoring the limitations of continuous-value error metrics for classifying drought versus non-drought events. By prioritizing classification skill (true/false event discrimination) over aggregate fit, the ROC–Youden framework provides an operationally relevant basis for dataset selection in agricultural drought monitoring. The SPI3-focused findings are directly applicable to short-term crop-water decisions, whereas SPI12 results inform longer-term water allocation.
References
Beharry, S.L., Gabriels, D., Lobo, D., & Clarke, R.M., (2019). A 35-year meteorological drought analysis in the Caribbean Region: case study of the small island state of Trinidad and Tobago. SN Applied Sciences, 1. https://doi.org/10.1007/s42452-019-1296-4.
Bhardwaj, J., Kuleshov, Y., Chua, Z.W., Watkins, A.B., Choy, S., & Sun, Q. (2022). Evaluating satellite soil moisture datasets for drought monitoring in Australia and the South-West Pacific. Remote Sensing, 14, 3971. https://doi.org/10.3390/rs14163971
Caribbean Climate Information Education and Tools (CLIE’nT). (n.d.). CARiDRO: Caribbean Assessment of Regional Drought. Available from: https://theclient.app/caridro
Ceglar, A., Moran-Tejeda, E., & Vicente-Serrano, S.M. (2012). Assessment of multi-scale drought datasets to quantify drought severity and impacts in agriculture: A case study for Slovenia. International Journal of Spatial Data Infrastructures Research, 7, 464–487. https://doi.org/10.2902/1725-0463.2012.07.art21
Centella, A., Bezanilla, A., Vichot, A., & Silva, M. (2014). User's guide CARiDRO: An online tool for drought assessment. INSMET, Cuba.
Centella, A., Bezanilla, A., Vichot, A., & Silva, M. (2017). CARiDRO: The Caribbean Assessment Regional Drought Tool. Case Study Report 1 of the CARIWIG Project. Institute of Meteorology, Cuba. Available from: https://cdkn.org/resource/case-study-caridro-caribbean-assessment-regional-drought-tool?
Farrell, D., Trotman, A. & Cox, C. (2010): Drought Impacts and Early Warning in The Caribbean, The Drought of 2009-2010. Global Assessment Risk Reduction, 22pp.
Gupta, H.V., Kling, H., Yilmaz, K.K., & Martinez, G.F. (2009). Decomposition of the mean squared error and NSE performance criteria: Implications for improving hydrological modelling. Journal of Hydrology, 377(1–2), 80–91. https://doi.org/10.1016/j.jhydrol.2009.08.003
Guttman, N.B. (1994). On the sensitivity of sample L moments to sample size. Journal of Climate, 7(6), 1026–1029.
Guttman, N.B. (1999). Accepting the standardized precipitation index: A calculation algorithm. Journal of the American Water Resources Association, 35(2), 311–322.
Hao, Z. & AghaKouchak, A. (2013). Multivariate Standardized Drought Index: A parametric multi-index model. Advances in Water Resources, 57: 12–18. https://doi.org/10.1016/j.advwatres.2013.03.009
Hayes, M., Svoboda, M., Wall, N., & Widhalm, M. (2011). The Lincoln Declaration on Drought Indices: Universal Meteorological Drought Index Recommended. Bulletin of the American Meteorological Society, 92(4), 485-488. https://doi.org/10.1175/2010BAMS3103.1
IPCC, (2013). Climate change 2013: The physical science basis. Contribution of Working Group I to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge University Press, Cambridge, UK and New York, USA.
IPCC, (2021). Climate Change 2021: The Physical Science Basis. Contribution of Working Group I to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change, Masson-Delmotte, V., P. Zhai, A. Pirani, S. L. Connors, C. P´ean, S. Berger, N. Caud, Y. Chen, L. Goldfarb, M.I. Gomis, M. Huang, K. Leitzell, E. Lonnoy, J.B.R. Matthews, T.K. Maycock, T. Waterfield, O. Yelekçi, R. Yu, and B. Zhou (eds.). Cambridge University Press, Cambridge, United Kingdom and New York, NY, USA, 2391 pp, https://doi.org/10.1017/9781009157896.
Junge, M.R., & Dettori, J.R. (2024). ROC solid: Receiver operator characteristic (ROC) curves as a foundation for better diagnostic tests. Global Spine Journal, 8(4), 424–429. https://doi.org/10.1177/2192568218778294
Kheyruri, Y., Sharafati, A., & Neshat, A. (2023). Predicting agricultural drought using meteorological and ENSO parameters in different regions of Iran based on the LSTM model. Stochastic Environmental Research and Risk Assessment, https://doi.org/10.1007/s00477-023-02465-6
Kling, H., Fuchs, M., & Paulin, M. (2012). Runoff conditions in the upper Danube basin under an ensemble of climate change scenarios. Journal of Hydrology, 424–425, 264–277. https://doi.org/10.1016/j.jhydrol.2012.01.011
Madrigal, J., Solera, A., Suárez-Almiñana, S., Paredes-Arquiola, J., Andreu, J., & Sánchez-Quispe, S.T. (2018). Skill assessment of a seasonal forecast model to predict drought events for water resource systems. Journal of Hydrology, 564, 574–587. https://doi.org/10.1016/j.jhydrol.2018.07.046
McKee, T.B., Doesken, N.J., & Kleist, J. (1993). Relationship of drought frequency and duration to time scales. Eighth Conference on Applied Climatology, 17–22 January 1993, Anaheim, CA, USA. Available from: www.droughtmanagement.info/literature/AMS_Relationship_Drought_Frequency_Duration_Time_Scales_1993.pdf
Miller, P.W., & Ramseyer, C.A. (2020). Did the Climate Forecast System Anticipate the 2015 Caribbean Drought? Journal of Hydrometeorology, 21(6), 1245-1258. https://doi.org/10.1175/JHM-D-19-0284.1
Mishra, A.K. & Singh, V.P. (2010) A Review of Drought Concepts. Journal of Hydrology, 391, 202-216. https://doi.org/10.1016/j.jhydrol.2010.07.012
Murphy, A.H. (1996). The Finley affair: A signal event in the history of forecast verification. Weather and Forecasting, 11(1), 3–20. https://doi.org/10.1175/1520-0434(1996)011<0003:tfaase>2.0.co;2
Nandgude, N., Singh, T. P., Nandgude, S., & Tiwari, M. (2023). Drought Prediction: A Comprehensive Review of Different Drought Prediction Models and Adopted Technologies. Sustainability, 15(15), 11684. https://doi.org/10.3390/su151511684
Nash, J.E., & Sutcliffe, J.V. (1970). River flow forecasting through conceptual models, part I - A discussion of principles. Journal of Hydrology, 10(3), 282–290. https://doi.org/10.1016/0022-1694(70)90255-6
National Drought Mitigation Center. (2018). SPI Generator [software]. University of Nebraska-Lincoln. Available from: https://drought.unl.edu/Monitoring/SPI/SPIProgram.aspx
Naumann, G., Barbosa, P., Garrote, L., Iglesias, A., & Vogt, J. (2014). Exploring drought vulnerability in Africa: An indicator-based analysis to be used in early warning systems. Hydrology and Earth System Sciences, 18, 1591–1604. https://doi.org/10.5194/hess-18-1591-2014
Orimoloye, I.R., Belle, J.A., Orimoloye, Y.M., Olusola, A.O., & Ololade, O.O. (2022). Drought: A common environmental disaster. Atmosphere, 13, 111. https://doi.org/10.3390/atmos13010111
Ortega-Gaucin, D., De la Cruz Bartolón, J., & Castellano Bahena, H. V. (2018). Drought Vulnerability Indices in Mexico. Water, 10(11), 1671. https://doi.org/10.3390/w10111671
Oyounalsoud, M.S., Abdallah, M., Yilmaz, A.G., Siddique, M., & Atabay, S. (2023). A new meteorological drought index based on fuzzy logic: Development and comparative assessment with conventional drought indices. Journal of Hydrology, 619, 129306. https://doi.org/10.1016/j.jhydrol.2023.129306
Paredes, F., & Guevara, E. (2010). Desarrollo y evaluación de un modelo para predecir sequías meteorológicas en los Llanos de Venezuela. Bioagro, 22(1), 3–10.
Paredes-Trejo, F., & Barbosa, H. (2017). Evaluation of the SMOS-derived soil water deficit index as agricultural drought index in Northeast of Brazil. Water, 9(6), 377. https://doi.org/10.3390/w9060377
Peres, D.J., & Cancelliere, A. (2014). Derivation and evaluation of landslide-triggering thresholds by a Monte Carlo approach. Hydrology and Earth System Sciences, 18(12), 4913–4931. https://doi.org/10.5194/hess-18-4913-2014
Peres, D.J., Iuppa, C., Cavallaro, L., Cancelliere, A., & Foti, E. (2015). Significant wave height record extension by neural networks and reanalysis wind data. Ocean Modelling, 94, 128–140. https://doi.org/10.1016/j.ocemod.2015.08.002
Trotman A., Joyette A., Van Meerbeeck C., Mahon R., Cox S., Cave N., and Farrell D. (2018). Drought Risk Management in the Caribbean Community: Early Warning Information and Other Risk Reduction Considerations. In: Wilhite, D. and Pulwarty, R. (Eds.) (2018). Drought and Water Crises. Boca Raton: CRC Press, https://doi.org/10.1201/b22009
Venturi, S., Dunea, D., Mateescu, E., Virsta, A., Petrescu, N., & Casadei, S. (2024). Drought monitoring in southeastern Romania based on the comparison and correlation of SPEI and SPI indices. Scientific Papers. Series E. Land Reclamation, Earth Observation & Surveying,Environmental Engineering. Vol. XIII, 2024 Print ISSN 2285-6064, CD-ROM ISSN 2285-6072, Online ISSN 2393-5138, ISSN-L 2285-6064.]
Vicente-Serrano, S. M., Beguería, S., & López-Moreno, J. I. (2010). A Multiscalar Drought Index Sensitive to Global Warming: The Standardized Precipitation Evapotranspiration Index. Journal of Climate, 23(7), 1696-1718. https://doi.org/10.1175/2009JCLI2909.1
Vicente-Serrano, S.M., Domínguez-Castro, F., Reig, F., Tomás-Burguera, M., Peña-Angulo, D., & Latorre, B. (2022). A global drought monitoring system and dataset based on ERA5 reanalysis: A focus on crop-growing regions. Geoscience Data Journal, 00, 1–14. https://doi.org/10.1002/gdj3.178
Wilhite, D.A. & Pulwarty, R. (Eds) (2018) Drought and Water Crises: Integrating Science, Management and Policy, 2nd edn. CRC Press, Boca Raton, FL, USA.
Wilhite, D.A., & Glantz, M. (1985). Understanding the drought phenomenon: The role of definitions. Water International, 10, 111–120.
World Meteorological Organization. (2012). Standardized precipitation index: User guide. WMO-No. 1090. ISBN 978-92-63-11091-6. Available from: https://library.wmo.int/doc_num.php?explnum_id=7768
World Meteorological Organization & Global Water Partnership. (2016). Handbook of drought indicators and indices (M. Svoboda & B. A. Fuchs). Integrated Drought Management Programme, Integrated Drought Management Tools and Guidelines Series 2, WMO-No. 1173. Geneva, Switzerland, and Stockholm, Sweden. ISBN 978-92-63-11173-9
Youden, W.J. (1950). Index for rating diagnostic tests. Cancer, 3, 32–35. https://doi.org/10.1002/1097-0142(1950)3:1<32::AID-CNCR2820030106>3.0.CO;2-3
Downloads
Published
How to Cite
Issue
Section
Categories
License
Copyright (c) 2026 ESEQUIEL ROLANDO JIMÉNEZ ESPINOSA , DONESSA DAVID
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
This is a human-readable summary of (and not a substitute for) the license. Disclaimer.
You are free to:
Share — copy and redistribute the material in any medium or format
Adapt — remix, transform, and build upon the material
The licensor cannot revoke these freedoms as long as you follow the license terms.
Under the following terms:
Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
NonCommercial — You may not use the material for commercial purposes.
ShareAlike — If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original.
No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
Notices:
You do not have to comply with the license for elements of the material in the public domain or where your use is permitted by an applicable exception or limitation.
No warranties are given. The license may not give you all of the permissions necessary for your intended use. For example, other rights such as publicity, privacy, or moral rights may limit how you use the material.
