Modernizing Probable Maximum Precipitation Estimation (2024)
Chapter: Appendix D: Criteria for a Modern PMP Estimation Process
Visit
NAP.edu/10766
to get more information about this book, to buy it in print, or to
download it as a free PDF.
Previous Chapter:
Appendix C: Dam Characteristics
Appendix D
Next Chapter: Appendix E: R Code used in Report Figures 3-5 and 5-3
Suggested Citation:
"Appendix D: Criteria for a Modern PMP Estimation Process." National Academies of Sciences, Engineering, and Medicine. 2024. Modernizing Probable Maximum Precipitation Estimation. Washington, DC: The National Academies Press.
doi: 10.17226/27460.
Appendix D
Criteria for a Modern PMP Estimation Process
This appendix summarizes the evaluation criteria developed by the committee for use in assessing the current and recommended approaches for estimating PMP. Table D-1 was introduced in Chapter 2, where the committee assessed the current PMP estimation process, demonstrating that the current process fails 20 of 23 criterion while partially meeting 3 necessary for producing objective, transparent, reproducible, and accessible PMP estimates, as well as key rarity (annual exceedance probability [AEP]) and uncertainty characterizations. In Chapter 5, the committee assessed the proposed near-term approach as meeting 12 criterion, partially meeting 9, and failing criterion related to AEP and uncertainty characterization, and the long-term approach as meeting 17 criterion including those for AEP estimation and uncertainty criterion, while rendering 6 as nonapplicable.
Suggested Citation:
"Appendix D: Criteria for a Modern PMP Estimation Process." National Academies of Sciences, Engineering, and Medicine. 2024. Modernizing Probable Maximum Precipitation Estimation. Washington, DC: The National Academies Press.
doi: 10.17226/27460.
TABLE D-1 User Criteria for Valid/Useful PMP Estimates and Estimation Process
| Criteria | Committee Assessment of Current Estimation Process | Recommended Near-term Process | Recommended Long-term Process |
|---|---|---|---|
| Data collection, characterization, and availability | |||
| Storm observations | |||
| Ongoing and systematically collected and geospatially- and temporally dense storm rainfall observations are employed | Fails: Most HMRs and more recent state-level PMPs are based on irregular and sparse observations of extreme precipitation. HMR 51, for example, though covering most of the nation east of the 105th meridian, was based on 55 storms that occurred from 1878 to 1972. Characterization of extreme precipitation during these storms and those used for state-level PMPs was based on sparse surface rain gauge networks greatly supplemented by “bucket surveys” following major storms. | Meets: The near-term recommended process includes use of conventional rain gauge and all other measurements that can be obtained for a storm, including bucket survey measurements when available, but relies principally on use of modern radar with a spatial resolution of approximately 1 km and a temporal resolution of 5-15 minutes. | Meets: The long-term recommended process incorporates data from reanalysis of historic storms and ongoing radar-based observations but relies mainly on large ensemble kilometer-scale modeling to supply storm data with which to construct PMP probability distributions. |
| Storm catalogs | |||
| Storm catalog is centralized, up-to-date, and publicly accessible | Fails: Some agencies and private sector entities have compiled storm catalogs, but maintenance and updates are irregular and driven by individual agency project or regulatory needs. Public accessibility various greatly. | Meets: The near-term recommended process includes digitization of existing (2023) PMP-scale storm data and construction of a national, centralized, and publicly accessible PMP storm catalog. | Not applicable: The long-term recommended process will substitute model simulations of extreme precipitation events for each gridded cell and watershed. |
| New storm observations are systematically included | Fails: Storm catalog updates are irregular and often do not include recent storms. | Meets: The near-term recommended process envisions ongoing reanalysis and characterization of recent storms captured by radar observations. | Meets: The long-term recommended process envisions ongoing reanalysis and characterization of recent storms captured by radar observations. |
Suggested Citation:
"Appendix D: Criteria for a Modern PMP Estimation Process." National Academies of Sciences, Engineering, and Medicine. 2024. Modernizing Probable Maximum Precipitation Estimation. Washington, DC: The National Academies Press.
doi: 10.17226/27460.
| Criteria | Committee Assessment of Current Estimation Process | Recommended Near-term Process | Recommended Long-term Process |
|---|---|---|---|
| Catalog routinely includes digitized rainfall fields to document and characterize temporal and geospatial intensity rainfall distributions | Fails: Most storms included in current storm catalogs include Depth-Area-Duration curves but lack gridded information. | Meets: The near-term recommended process envisions ongoing digitization of recent storms captured by radar observations. | Meets: The long-term recommended process envisions ongoing digitization of recent storms captured by radar observations. |
| Modeling process (screening, adjustment, simulation, and statistical characterization to produce a PMP estimate) | |||
| Storm maximization | |||
| Maximization of observed or transposed storms is objective and reproducible | Fails: Storm and long-term precipitable water estimates were based on surface dew point observations and empirical vertical moisture distribution models requiring subjective estimates of model parameters. | Partially meets: Process-based storm models are now available that could provide storm maximized precipitation and simulated temporal histories at high geospatial resolutions without resort to empirical techniques. The recommended near-term actions include development of a model validation project to identify and test promising model approaches. | Not applicable: The long-term recommended process recommends numerical simulations that include storms covering a broad range of extreme-precipitation events that will not require traditional individual storm maximization techniques. However, realizing this recommendation will require substantial and sustained model development and evaluation. |
| Maximization takes into account known interactions between major dynamical and thermodynamical processes | Fails: Current estimates of PMP are based on empirical models that inaccurately assume the independence of dynamical and thermodynamical process that has not withstood critical examination. | Partially meets: Process-based storm models are now available that could provide storm maximized precipitation and simulated temporal histories at high geospatial resolutions without resort to empirical techniques. The recommended near-term actions include development of a model validation project to identify and test promising model approaches. | Not applicable: The long-term recommended process recommends numerical simulations that include storms covering a broad range of extreme-precipitation events that will not require traditional individual storm maximization techniques. However, realizing this recommendation will require substantial and sustained model development and evaluation. |
Suggested Citation:
"Appendix D: Criteria for a Modern PMP Estimation Process." National Academies of Sciences, Engineering, and Medicine. 2024. Modernizing Probable Maximum Precipitation Estimation. Washington, DC: The National Academies Press.
doi: 10.17226/27460.
| Criteria | Committee Assessment of Current Estimation Process | Recommended Near-term Process | Recommended Long-term Process |
|---|---|---|---|
| Storm transposition | |||
| Transposition regions are developed and defined objectively | Fails: HMR PMP estimates often employ transposition regions that are based on subjective meteorological judgments and driven by use of singularly unique historic storms within sparse datasets. | Partially meets: The recommended near-term program includes development of transposition guidelines and tools and a model development and evaluation program to improve transposition tools and techniques. | Not applicable: The long-term recommended process recommends numerical simulations that include storms for every cell and watershed in a high-resolution grid that will not require traditional storm-transposition. However, realizing this recommendation will require substantial and sustained model development and evaluation. |
| Transposition adjustments are objectively determined | Fails: HMR PMP estimates are often based on correction factors that have little theoretical bases. | Partially meets: The recommended near-term program includes development of transposition guidelines and tools and a model development and evaluation program to improve transposition tools and techniques. | Not applicable: The long-term recommended process recommends numerical simulations that include storms for every cell and watershed in a high-resolution grid that will not require traditional storm-transposition. However, realizing this recommendation will require substantial and sustained model development and evaluation. |
| State of science analytical procedures are employed | Partially meets: The HMRs employed smooth enveloping isohyets drawn to transposed observed and maximized storm precipitation values for various durations and area extents. The positioning and spacing of the isohyets were refined through consideration of regional information. | Partially meets: The recommended near-term program includes development of transposition guidelines and tools. | Not applicable: The long-term recommended process recommends numerical simulations that include storms for every cell and watershed in a high-resolution grid that will not require traditional storm-transposition. However, realizing this recommendation will require substantial and sustained model development and evaluation. |
Suggested Citation:
"Appendix D: Criteria for a Modern PMP Estimation Process." National Academies of Sciences, Engineering, and Medicine. 2024. Modernizing Probable Maximum Precipitation Estimation. Washington, DC: The National Academies Press.
doi: 10.17226/27460.
| Criteria | Committee Assessment of Current Estimation Process | Recommended Near-term Process | Recommended Long-term Process |
|---|---|---|---|
| Model fidelity (for this evaluation: the degree to which a model or simulation reproduces a state or behavior of a natural process) | |||
| Strong correlation with natural extreme rainfall generation processes through physically based analogs and algorithms | Fails: Past PMP estimation was based on empirical methods that estimated reasonable upper limits of maximized observed and transposed storm data with limited correlation to natural processes. | Partially meets: The recommended near-term program includes a model evaluation project to identify and integrate appropriate model algorithms for simulation of extreme storm events and their maximization, transposition, and envelopment pending transition to the recommended long-term process. | Meets: The recommended long-term program envisions the use of selected, data-driven, process-verified model algorithms and ongoing research to improve these models and apply them to varying storm types and climatic conditions. |
| Modern modeling (for this evaluation: use of physically based algorithms that accurately simulate long ensembles of extreme rainfall events across most storm-types of concern and applicable climates relevant to PMP estimation and capable of outputting gridded results) | |||
| Use of up-to-date climate and weather models capable of accurately simulating long ensembles of extreme precipitation events of varying storm types and climatic conditions | Fails: Past PMP estimation was based on empirical methods that estimated reasonable upper limits of maximized observed and transposed storm data with limited correlation to natural processes. | Partially meets: The recommended near-term program includes a model evaluation project to identify and integrate appropriate model algorithms for simulation of extreme storm events and their maximization, transposition, and envelopment pending transition to the recommended long-term process. | Meets: The recommended long-term program envisions the use of selected, data-driven, process-verified model algorithms to simulate large-ensemble, high-resolution extreme storm rainfall events with which to construct PMP distributions. |
| Character of PMP and related products | |||
| High-resolution, grid-based products | |||
| GIS datasets depicting high-resolution (1-km spatial and 60-minute temporal) gridded PMP rainfalls for selected durations | Partially meets: Early PMP studies provided paper and then digitalized maps of PMP totals for selected storm durations. More recent, private-sector based studies have provided geospatial estimates for selected storm-durations together with storm catalog information. |
Partially meets: The near-term recommended process includes digitization of existing (2023) PMP-scale storm data and identification and digitization of subsequent storms based on radar information. | Meets: The long-term recommended process includes numerical simulations of simulated storms that can be delivered as grid-based products. |
Suggested Citation:
"Appendix D: Criteria for a Modern PMP Estimation Process." National Academies of Sciences, Engineering, and Medicine. 2024. Modernizing Probable Maximum Precipitation Estimation. Washington, DC: The National Academies Press.
doi: 10.17226/27460.
| Criteria | Committee Assessment of Current Estimation Process | Recommended Near-term Process | Recommended Long-term Process |
|---|---|---|---|
| Probability estimates | |||
| AEP estimates of PMP for selected durations | Fails: The current PMP definition does not permit development of formal AEP estimates, thus no PMP study has provided AEP estimates. | Fails: The near-term recommended program will continue to operate based on the historic definition and will not produce AEP estimates. | Meets: The adoption of the recommended PMP definition will permit the association of PMP events with AEPs through long-term simulations that produce large numbers (100,000s) of precipitation events, including PMP-scale precipitation events, that will be analyzed statistically to estimate depths associated with and lead to PMP-magnitude AEPs. |
| Uncertainty | |||
| Reliable uncertainty estimates | Fails: Subjective assessments have been provided by some PMP studies. | Fails: The current PMP definition does not permit development of formal uncertainty estimates. | Meets: The adoption of the recommended PMP definition will permit the characterization of uncertainty in PMP estimates. |
| Update frequency | |||
| Every 10 years or as often as underlying conditions change significantly (Provide process for quality-assuring and incorporating user-agency updates when resources for national updates are not available) |
Fails: HMRs and more recent state-based updates have been irregular and ad-hoc as they are largely funded through state agencies. | Partially Meets: The recommended near-term process envisions ongoing capture of extreme events using radar-based observations and near-continuous reanalysis of these storms as the data are made available. Recommended approach facilitates updates needed to accommodate new data or climate change. | Meets: The recommended long-term process envisions ongoing capture of extreme events using radar-based observations and near-continuous reanalysis of these storms as the data are made available. Recommended approach would facilitate updates needed to accommodate new data or climate change. |
| Applicability of PMP Products | |||
| Geospatial extent of covered area | |||
| National extent without state-line or regional faults | Fails: HMRs and state-based products applied to discrete regions to cover most of continental U.S. for varying time periods. | Meets: The recommended near-term program includes requirements to meet this criterion. | Meets: The recommended long-term program includes requirements to meet this criterion. |
Suggested Citation:
"Appendix D: Criteria for a Modern PMP Estimation Process." National Academies of Sciences, Engineering, and Medicine. 2024. Modernizing Probable Maximum Precipitation Estimation. Washington, DC: The National Academies Press.
doi: 10.17226/27460.
| Criteria | Committee Assessment of Current Estimation Process | Recommended Near-term Process | Recommended Long-term Process |
|---|---|---|---|
| Spatial resolution | |||
| 1 to 10,000 square miles | Fails: HMRs generally provided estimates for 10–20,000 square miles. State-based products differ in resolution. | Meets: The recommended near-term program includes requirements to meet this criterion | Meets: The recommended long-term program includes requirements to meet this criterion. |
| Storm duration | |||
| 1 hour to 7 days | Fails: HMRs generally provided estimates for 6–72 hours duration. | Meets: The recommended near-term program includes requirements to meet this criterion | Meets: The recommended long-term program includes requirements to meet this criterion. |
| Public Accountability and the PMP Estimation Process | |||
| Transparency (for this evaluation: the ease with which citizens can access governmental information and assess agency adherence to established timelines, rules, procedures, protocols, and standards of practice and care in governmental decision making) | |||
| Consistent and planned process disclosed prior to study with follow-on adherence to the plan timelines and | Fails: Early studies driven by varied agency needs and resources and conducted with little public input. Source data and models are often not well documented or | Meets: The near-term recommended program includes establishment of tracked timelines and written procedures to guide selection | Meets: The long-term recommended program includes establishment of tracked timelines and written procedures to guide selection of data input, |
| established procedures disclosing data sources, input and output data, technical assumptions, computer scripts and parameters, and ancillary information | easily accessed. Recent state-level studies have involved greater transparency and significant state-level interagency oversight. | of data input, computational processes, and software, and publish interim and final results and the use of periodic public notices through NOAA publications, the Federal Register, and other official means to document progress, explain delays or new directions, and to seek public comment. | computational processes, and software, and publish interim and final results and the use of periodic public notices through NOAA publications, the Federal Register, and other official means to document progress, explain delays or new directions, and to seek public comment. |
Suggested Citation:
"Appendix D: Criteria for a Modern PMP Estimation Process." National Academies of Sciences, Engineering, and Medicine. 2024. Modernizing Probable Maximum Precipitation Estimation. Washington, DC: The National Academies Press.
doi: 10.17226/27460.
| Criteria | Committee Assessment of Current Estimation Process | Recommended Near-term Process | Recommended Long-term Process |
|---|---|---|---|
| Objectivity | |||
| Minimize procedural reliance on subjective judgment in the selection of source data and methods and in the absence of clearly objective measures channel selections with decision support matrices | Fails: Manual review and selection of storms, maximization techniques, transposition extent and other inputs and procedures were often based on subjective assessments. | Meets: The near-term recommended program envisions written procedural guidance and decision support matrices to proscribe and limit subjective influences. | Meets: The long-term recommended program envisions written procedural guidance and decision support matrices to proscribe and limit subjective influences. |
| Accessibility | |||
| Access to data, computer scripts, and interim and final results is provided through public websites and open publications | Partially meets: The various HMRs and state-based PMP products vary in providing public access to source data or codes some of which are proprietary. | Meets: The near-term recommended program envisions the establishment of public webpages to supply data, computer scripts, and other inputs and outputs used in estimation of PMPs. | Meets: The long-term recommended program envisions the establishment of webpages to supply data, computer scripts, and other inputs and outputs. |
| Reproducibility | |||
| PMP estimates are fully reproducible using the same inputs, tools, parameters, and settings. PMP estimates obtained by independent practitioners are consistent | Fails: Generally consistent high-level process but with varying data quantity and quality due to reliance on opportunistic data collection and haphazard documentation, somewhat subjective storm selection criteria, and varying adjustments for topographic effects. | Meets: The near-term recommended program envisions written procedural guidance and decision support matrices to proscribe and limit subjective influences and the release of all input data and tools and publication of model parameters and settings. | Meets: The near-term recommended program envisions written procedural guidance and decision support matrices to proscribe and limit subjective influences and the release of all input data and tools and publication of model parameters and settings. |
Suggested Citation:
"Appendix D: Criteria for a Modern PMP Estimation Process." National Academies of Sciences, Engineering, and Medicine. 2024. Modernizing Probable Maximum Precipitation Estimation. Washington, DC: The National Academies Press.
doi: 10.17226/27460.
| Criteria | Committee Assessment of Current Estimation Process | Recommended Near-term Process | Recommended Long-term Process |
|---|---|---|---|
| Collaboration | |||
| Development of procedures and production of PMP estimates are produced through sustained collaboration to the extent possible | Fails: Past development of PMP estimates were conducted largely within the confines of agency structures with little public input. Recent state-level PMP estimates have been vetted with external expert panels, but without upfront participation of private sector or environmental interest representatives. | Meets: The near-term recommended program envisions ongoing engagement of the scientific and practitioner communities. | Meets: The long-term recommended program envisions ongoing engagement of the scientific and practitioner communities. |
Suggested Citation:
"Appendix D: Criteria for a Modern PMP Estimation Process." National Academies of Sciences, Engineering, and Medicine. 2024. Modernizing Probable Maximum Precipitation Estimation. Washington, DC: The National Academies Press.
doi: 10.17226/27460.
This page intentionally left blank.
Subscribe to Email from the National Academies
Keep up with all of the activities, publications, and events by subscribing to free updates by email.