Home
A JFSP Fire Science Exchange Network
Bringing People Together & Sharing Knowledge in the Northern Rockies

Commentary on the article “Burn probability simulation and subsequent wildland fire activity in Alberta, Canada - Implications for risk assessment and strategic planning” by J. L. Beverly and N. McLoughlin

Author(s): Marc-Andre Parisien, Alan A. Ager, Ana M. G. Barros, Denyse A. Dawe, Sandy Erni, Mark A. Finney, Charles W. McHugh, Carol Miller, Sean A. Parks, Karen L. Riley, Karen C. Short, Christopher A. Stockdale, Xianli Wang, Ellen Whitman
Year Published: 2020
Description:

Monte Carlo simulations using wildland fire spread models have been conducted to produce numerical estimates of fire likelihood, project potential fire effects, and produce event sets of realistic wildfires (Parisien et al., 2019). The application of these methods has greatly expanded over the last few decades as a result of increased computation capabilities, available data, and our fundamental understanding of landscape fire dynamics. In their recently published article, Beverly and McLoughlin (2019) attempt to assess the accuracy of fire likelihood outputs (hereafter “burn probability” [BP]) they produced for five large areas in Alberta, Canada, by testing the “correspondence between observed burned areas and pre-fire burn probability maps” in order to “explore how the expectations of decision-makers influenced our assessment of map accuracy.” To do so, they superimposed BP estimates computed with the Burn-P3 fire simulation model (Parisien et al., 2005) with areas recently burned by wildfire from the reference year used in Burn-P3 until 2017. Their results show a moderate statistical preference of recent burns for high-probability areas in three study areas and no preference in the remaining two. This leads them to conclude that “the use of these maps for research or other applications should be approached with caution and consideration of their shortcomings and apparent limitations.” In dismissing the accuracy of the BP estimates, they are sending an erroneous message to managers and the research community that BP maps are not useful.

We agree with Beverly and McLoughlin (hereafter B & M) that assessing the skill of BP models—or any model—is necessary, but we argue that the manner in which their evaluation was executed undermines (or even invalidates) some of their conclusions regarding the utility and potential accuracy of fire simulation model outputs. We display and discuss three major weaknesses in B & M’s analysis: (1) the inappropriate method of assessing accuracy, (2) a misinterpretation of BP outputs, and (3) data quality considerations. Finally, we propose ways in which a discussion on assessing BP model skill could move forward.

Citation: Parisien, Marc-Andre; Ager, Alan A.; Barros, Ana M.; Dawe, Denyse; Erni, Sandy; Finney, Mark A.; McHugh, Charles W.; Miller, Carol; Parks, Sean A.; Riley, Karin L.; Short, Karen C.; Stockdale, Christopher A.; Wang, Xianli; Whitman, Ellen. 2020. Commentary on the article “Burn probability simulation and subsequent wildland fire activity in Alberta, Canada - Implications for risk assessment and strategic planning” by J. L. Beverly and N. McLoughlin. Forest Ecology and Management. 460: 117698. https://doi.org/10.1016/j.foreco.2019.117698
Topic(s): Fire Behavior, Simulation Modeling, Risk
Ecosystem(s): None
Document Type: Book or Chapter or Journal Article
NRFSN number: 22490
Record updated: Jan 13, 2021