The U.S. Environmental Protection Agency’s (EPA) Event Detection System (EDS) Challenge research project was initiated to advance the state of knowledge in the field of water quality event detection.  The objectives included:

• Identifying available EDSs and exploring their capabilities  
• Providing EDS developers a chance to train and test their software on a large quantity of data – both raw utility data and simulated events  
• Pushing the WQM data analysis field forward by challenging developers to optimize their EDSs and incorporate innovative approaches to WQM data analysis
• Developing and demonstrating a rigorous procedure for the objective analysis of EDS performance, considering both invalid alerts and detection rates  
• Evaluating available EDSs using an extensive dataset and this precise evaluation procedure 

This was a research effort.  An objective was not to identify a “winner.”   

 

Five EDSs were voluntarily submitted for this study: 

• CANARY - Sandia National Laboratories, EPA
• ana::tool - s::can
• OptiEDS - OptiWater (Elad Salomons)
• BlueBoxTM - Whitewater Security
• Event Monitor - Hach Company 

This report begins with an overview of the EDS Challenge, including the methodology and data used for testing.  Section 4 analyzes EDS performance.  Section 4.2 summarizes the detected events and invalid alerts produced by each EDS, considering both their raw binary output (Section 4.2.1) and alternate performance that could be achieved by modifying the alert threshold setting (Section 4.2.2).  Section 4.3 investigates the impact of the simulated contamination characteristics (such as the contaminant used) on event detection across all EDSs. 

 

Section 5 presents findings and conclusions from the EDS Challenge, including the following:  

• WQ event detection can provide valuable information to utility staff.   
• There is no “best” EDS.   
• The ability of an EDS to detect anomalous WQ strongly depends on the “background” WQ variability of the monitoring location.  The characteristics of the WQ change also impacts the ability of an EDS to detect it.   
• Changing an EDS’s configuration settings can significantly impact alerting.  In general, reconfiguration to reduce invalid alerts reduces the detection sensitivity as well.  

This report concludes with ideas for future research in this area and a discussion of practical considerations for utilities when considering EDS implementation.