Uncertainty in Complex Event Processing Systems

Projects
UCEP

Uncertainty in Complex Event Processing Systems

Overview

Complex Event Processing (CEP) is a powerful technology for analyzing streams of information about real-time distributed events, coming from different sources, and for extracting conclusions from them. CEP permits hierarchically defining complex events based on the primitive events produced by the incoming sources, or from other complex events, in order to identify elaborated situations of interest, and to quickly respond to them. However, in many situations these events are not free from uncertainty, due to either unreliable data sources and networks, measurement uncertainty, or to the inability to determine whether an event has actually happened or not. We discuss how CEP systems can incorporate different kinds of uncertainty, both in the events and in the patterns.

In this companion website to our works [2,3], the complete case studies and software artifacts we reference in our papers are available for download.

Case Study 1: Smart House

This system simulates a smart house, with sensors that permit detecting three basic parameters: temperature, carbon monoxide (CO) level, and whether the main door is open or not. These values are periodically sensed and notified by means of CEP events, as well as the position of the people living in the house. By analysing these primitive events, a set of CEP patterns (rules) are in charge of detecting several complex events such as:

TempIncrease: The temperature of the house has increased 2 or more degrees in less than one minute.
TempWarning: There are 4 TempIncrease events is less than 5 minutes, and the temperature is always above 33 degrees.
COHigh: CO levels exceed 5000 units.
FireWarning: A COWarning event is detected, followed by a TempWarning event within less than 5 seconds.
NobodyHome: The main door of the house is closed and there is nobody within the perimeter of the house.
CallFireDept: A FireWarning event occurs after a NobodyHome event is detected.

In this case study we show how it is possible to represent different kinds of uncertainty in the CEP events and in the patterns. The complete description of the case study is detailed in [3].

The file with the rules of our case study in plain Esper EPL [1] is available for download here and the other case, considering uncertainty, here. The Java projects as well as the input data set we have used to execute our experiments are available in our Git repository.

Case Study 2: Marathon

Let us suppose the runners of a marathon are equipped with sensors that generate events every time the runner passes one of the control points established every 5 Km. We are interested in the following situations, which should generate the corresponding complex events:

Escaped: When the passing time at one of the control points between a runner and the next one exceeds 30 seconds.
TheWall: The passing time of a runner between Km 35 and Km 30 is greater than her passing time between Km 30 and 25, plus 1 minute.
NegativeSplit: The time taken by a runner in the second half marathon is less than the time taken to complete the first one.
Optimal: A runner finishes the race with a negative split and without going through the wall.
Cheated: A runner has cheated if he finishes the marathon but has missed the control point at Km 21.097 (half marathon).
Record: A runner finishes the race in less than 2 hours, 3 minutes and 38 seconds.

The input datasets exceeding 100Mb could not be uploaded to GitHub due to GitHub's file size limit. These files can be downloaded from here:
marathon_2018_uncertainty_x10.csv
marathon_2018_uncertainty_x15.csv
marathon_2018_uncertainty_x25.csv
marathon_2018_uncertainty_x100.csv
marathon_2018_1000_x1000.csv
marathon_2018_x10.csv
marathon_2018_x15.csv
marathon_2018_x25.csv
marathon_2018_x100.csv
marathon_2018_1000_x250.csv
marathon_2018_inc_1000_x1000.csv

Case Study 3: Motorbike

Imagine a fleet of motorbikes equipped with sensors that produce real-time information about their state, such as the pressure of their two tires, their location and speed. They even sense whether the driver is on the seat or not. We are interested in monitoring them, being able to detect flat tires, vehicle crashes or accidents in real time. In particular, we are interested in the following complex events:

BlowOutTire: The pressure of one of the tires of a moving motorbike goes down from more than 2.0 BAR to less than 1.2 BAR in less than 5 seconds.
Crash: The speed of a motorbike goes from more than 50 km/h to 0 km/h in less than 3 seconds.
DriverLeftSeat: The seat sensor detects that the motor- bike driver has left the seat.
Accident: A moving motorbike suffers a blow out of one of its tires, then a Crash event is detected, and the driver is thrown out, everything within less than 3 seconds.
AccidentsReport: A complex event with the number of Accidents per day and location.
DangerousLocation: This event is raised every time 100 events of type Crash are detected in a given location.

The file with the rules of our case study in plain Esper EPL [1] is available for download here and the other case, considering uncertainty, here.The Java projects as well as the input data set we have used to execute our experiments are available in our Git repository.

The input datasets exceeding 100Mb could not be uploaded to GitHub due to GitHub's file size limit. These files can be downloaded from here:
motorbike_x50.csv
motorbike_x100.csv
motorbike_uncertainty_x50.csv
motorbike_uncertainty_x100.csv
motorbike_inc_1000_x1000.csv

Case Study 4: Air Quality

This real example models the air sensor network for controlling air quality in the Andalusian region. The network is composed of several sensor stations that gather the air pollutant measurements and send them to a main server. Each station has more or less sensors in charge of measuring the following pollutants: PM2.5, PM10, CO, O3, NO2 and SO2. For each pollutant previously mentioned (6 in total), we need to be warned when the following 7 event patterns are detected by the Esper CEP engine:

Avg: It computes in 6-min sliding windows the aggregate value (average) for the pollutant readings, grouping the information by each station.
Good: It detects pollutant readings whose values are considered as ``good'' for health.
Moderate: It detects pollutant readings whose values are considered as ``moderate'' for health
UnhealthyForSensitiveGroups: It detects pollutant readings whose values are considered as unhealthy for certain sensitive groups of people.
Unhealthy: It detects pollutant readings whose values are considered as unhealthy for people in general.
VeryUnhealthy: It detects pollutant readings whose values are considered as very unhealthy for people in general.
Hazardous: It detects pollutant readings whose values are considered as hazardous for health.

The correctness of the system patterns was already analysed in our previous work [4] using Maude. The file with the rules of our case study in plain Esper EPL [1] is available for download here and the other case, considering uncertainty, here. The Java projects as well as the input data set we have used to execute our experiments are available in our Git repository.

The input datasets exceeding 100Mb could not be uploaded to GitHub due to GitHub's file size limit. These files can be downloaded from here:
dec2016-apr2017-AirMeasurement.csv
dec2016-apr2017-AirMeasurement_x2.csv
dec2016-apr2017-AirMeasurement_x10.csv
QA_1000_x1000.csv
dec2016-apr2017-AirMeasurement_incertidumbre.csv
dec2016-apr2017-AirMeasurement_uncertainty.csv
QA_inc_1000_x1000.csv

Case Study 5: Tunnel Ventilation System

Let us assume a Tunnel Ventilation System (TVS) constantly monitored by several sensors to detect possible failures, such as obstructions. In usual setups sensors are evenly distributed along the tunnel and they measure the temperature and the concentration of oxygen in air. Now, consider a CEP application designed to detect TVS malfunctioning. Such application has to recognize critical situations starting from the raw data measured by sensors. Depending on the environment, the application requirements, and the user preferences, the presence of a TVS malfunctioning can be specified in many different ways. Here we present three possible definitions of the TVS malfunctioning event:

There is a TVS malfunctioning when there is an oxygen concentration lower than 18%, while a temperature higher than 30 was detected in the same sector of the tunnel in the previous 5min. The malfunctioning notification has to embed the sector of the tunnel, together with the oxygen concentration and temperature actually measured.
There is a TVS malfunctioning when there is a temperature higher than 30 in absence of a traffic jam in the same tunnel sector.
There is a TVS malfunctioning when there is an oxygen concentration lower than 18% and the average temperature in the last 5 min, in the same tunnel sector, was higher than 30.

This case study was initially proposed in [5]. The file with the rules of our case study in plain Esper EPL [1] is available for download here and the other case, considering uncertainty, here. The Java projects as well as the input data set we have used to execute our experiments are available in our Git repository.

The input datasets exceeding 100Mb could not be uploaded to GitHub due to GitHub's file size limit. These files can be downloaded from here:
OTT_x10000.csv
OTT_uncertainty_x1000.csv
OTT_uncertainty_x10000.csv

References

[1] EsperTech: Esper - Complex Event Processing.
[2] Nathalie Moreno, Manuel F. Bertoa, Gala Barquero, Loli Burgueño, Javier Troya, Adrián García-López, Antonio Vallecillo: Managing Uncertain Complex Events in Web of Things Applications. In Proc. of ICWE 2018. LNCS#10845, pp. 349-357, Springer, 2018
[3] Nathalie Moreno, Manuel F. Bertoa, Loli Burgueño, Antonio Vallecillo: Managing Measurement Uncertainty in CEP Systems, 2018. Submitted.
[4] Loli Burgueño, Juan Boubeta-Puig, Antonio Vallecillo: Formalizing Complex Event Processing Systems in Maude. IEEE Access 6: 23222-23241 (2018)
[5] Gianpaolo Cugola, Alessandro Margara, Matteo Matteucci, Giordano Tamburrelli. Introducing uncertainty in complex event processing: model, implementation, and validation. Computing 97 (2) (2015) 103–144.