Notifications

• Introduction
• Entity aggregation
• Basic notifications
  • Service logic
• User Messaging
  • Service

Introduction

Notification services can be implemented by dedicated modules. If implemented, notification services should follow the design pattern:

• No direct access to DB: There is no direct access from the service to the database using SQL. Read or update operations are performed using REST calls against the microservice API.
• Windows service deployment: Notification services are deployed as worker services - using cross platform options is encouraged (systemd on UNIX, windows service on Windows), logging, etc. needs to include an abstraction layer supporting booth deployment scenarios.
• Message distribution: the DCP framework is taking care of message distribution, underlying services emit to the MessageQueue

Entity aggregation

In order to combine entities from multiple microservices, GUID are used instead of Integers as unique Ids for the database. The workflow, for entity aggregator, is defined as follows:

• One API endpoint per microservice providing the resources is called. Every endpoint returns different types of notifications (e.g. Model, single value and multiple values).
• On the first call, notifications are combined in the same NGRX State array. This is iterated to build the UI. If some of the API calls fail or do not return anything, there is no change in the store array and successful requests data is still presented.
• On following API calls for notifications, the notifications in NGRX state is updated based on ID.

Basic notifications

The Basic Notification Service is built as a Windows service. Notification services can be differentiated between batch and continuous notifications.

• Continuous notifications are checked no matter if an event (batch) context is available or not.
• Batch notifications are only checked if a dedicated context is currently active

Service logic

The only state which is across multiple cycles is the timestamp of the first occurrence, in order to implement an alarm delay time. The general pattern of the continuous notifications is illustrated in the figure below:

stateDiagram-v2
    [*] --> Request_Data
    DataSource --> Request_Data
    Request_Data --> DataSource
    Request_Data --> for_value_In_data: Group notification sensors per data source
    state for_value_In_data{
        Notification_Conditions_Met --> TimestampFirstOccurence_has_value: Yes
        Notification_Conditions_Met --> TimestampFirstOccurence_has_value: Yes
        TimestampFirstOccurence_has_value
        TimestampFirstOccurence_has_value --> TimestampFirstOccurrence=NULL: No
        TimestampFirstOccurence_has_value --> TimestampFirstOccurrence=ObsId: No
        TimestampFirstOccurrence=ObsId --> Calculate_timediff
        Calculate_timediff --> TimeDiff>AlarmDelayTime
        TimeDiff>AlarmDelayTime --> Add_to_email_queue

    } 

The Basic module implements continuous and batch notifications for single and equations. The general design pattern is always the same: the DCP database contains whole expressions, e.g. "@sensorWebId@ > 6". During every cycle, the @sensorWebId@ parts are replaced with the actual values from the data source. After the replacement, an expression evaluating to true or false is present, which can be used to check the notification conditions.

This service runs a timed loop with an execution cycle of 60 seconds. Within every loop execution, the following steps are performed:

• The notification values are checked
  • Grouping the notifications by the data source and request compressed data
  • Check the returned data array for the notifications
  • Updated the TimestampFirstOccurence

Single and Equation notifications have independent check cycles, as data grouping is performed slightly different. For single parameters, only compressed/ data is requested in order to reduce the load on the underlying systems. The cycle as described above is executed. For equation notifications the request has to be done a little different as aligned timestamps for the sensor values are required in order to calculate the expression. Therefore, a nested request is needed in the first sub-request all the timestamps when values were recorded (indicating a change of values). With the 2nd sub-request interpolated values at all recorded timestamps are requested. This ensures that all events are safely evaluated in the right time context.

Notification records

erDiagram
    Notification ||--o{ NotificationEquation : owns
    Notification ||--o{ NotificationSingleParameter : owns
    Notification ||--|| NotificationStatusCheck : uses
    Notification ||--|{ NotificationSubscribers : sends

    Notification  {
        uniqueidentifier(16) ID "UniqueId"
        nvarchar(500) Title "User defined title for better identification"
        nvarchar Description "User definied detailed description"
        tinyint(1) Type "Enumeration for the type"
        nvarchar Filter "Global filter path to the notification to which the element is assigned"
        tinyint(1) Status "Enumeration for the status"
        datetime(8) ActivatedAt "Datetime of the (last) activation time"
        nvarchar(500) DeviceWebId "DeviceId on the datasource, used for retriving the batch context"
        int(4) SiteId "Relation to the site where the notification is linked to"
        tinyint(1) NotificationContext "Enumeration to identify the batch context behavior"
        tinyint(1) NotifcationCreationType "Flag, how the notification was created (user/system)"
        nvarchar Conditions "JSON object specifying the eventframe attribute conditions when to perform the checks"
        datetime2(8) SysStartTime 
        datetime2(8) SysEndTime 
        int(4) LastModifiedBy 
        int(4) OwnerUserId 
    }

    NotificationEquation  {
        int(4) ID "UniqueId"
        uniqueidentifier(16) NotificationId "Relation to notification for title, description and other basic information"
        nvarchar(100) ConditionType "Relation to notification for title, description and other basic information"
        nvarchar(500) ConditionValue "Value to compare against, NULL if compared using ranges"
        nvarchar Expression "Expression evalated during the check, sensor references in @, numbers in #"
        nvarchar(500) RangeConditionLowerValue "Lower bound value of the range, to compare against"
        nvarchar(500) RangeConditionUpperValue "Upper bound value of the range, to compare against"
        nvarchar(500) AlarmDelay "The alarm delay time in seconds"
        datetime(8) TimeStampFirstOccuren "Datetime of the notification evaluating to true in UTC"
        datetime2(8) SysStartTime 
        datetime2(8) SysEndTime 
        int(4) LastModifiedBy 
        int(4) OwnerUserId 
    }

    NotificationSingleParameter  {
        int(4) ID "UniqueId"
        uniqueidentifier(16) NotificationId "Relation to notification for title, description and other basic information"
        nvarchar(500) SensorWebId "UniqueId of the sensor on the datasource"
        nvarchar(100) ConditionType "Condition type used for the notification, can be greater, less, etc."
        decimal(9) ConditionValue "Value to compare against, NULL if compared using ranges"
        decimal(9) RangeConditionLowerValue "Lower bound value of the range, to compare against"
        decimal(9) RangeConditionUpperValue "Upper bound value of the range, to compare against"
        nvarchar(500) AlarmDelay "The alarm delay time in seconds"
        datetime(8) TimeStampFirstOccuren "datetime of the notification evaluating to true in UTC"
        datetime2(8) SysStartTime 
        datetime2(8) SysEndTime 
        int(4) LastModifiedBy 
        int(4) OwnerUserId 
    }

    NotificationStatusCheck  {
        int(4) ID "UniqueId"
        uniqueidentifier(16) NotificationId "Relation to the notification"
        nvarchar EventFrameId "Unique id on the datasource for the active eventframe"
        tinyint(1) BatchMachingConditions "Enumeration, showing the match status between the attribute values of the current active eventframes and the defined conditions"
    }

    NotificationSubscribers  {
        int(4) ID "UniqueId"
        uniqueidentifier(16) NotificationId "Relation to the notifications"
        int(4) UserId "Relation to the userId of the notification subscriber"
        datetime2(8) SysStartTime 
        datetime2(8) SysEndTime 
        int(4) LastModifiedBy 
        int(4) OwnerUserId 
    }

Records classification and audit trail

User Messaging

For some functionality e.g. notifications, etc. end users need to be notified via a message channel. DCP Framework provides with a single service for sending messages to end users. The services currently uses E-mail as the delivery channel. All modules can send messages to end-users utilizing the internal message bus. The service will handle the delivery to the end-users.

Service

Email Processor Service has the single responsibility for sending e-mails. All services/components can add messages to the CoreEmailQueue message queue. The e-mail processor service is reading from this queue and responsible for sending the messages with the exception handling in place. Historical data of all e-mail are stored with additional information and unhandled exceptions is stored in the database for debug and audit purposes.

Message Format

Templates E-mail templates are using HTML and CSS for formatting. Templates for HTML (user facing e-mails) are part of the backend project. To allow rendering in the Google Mail inbox no external libraries are used and images are embedded as base64 string. The templates contain placeholders, enclosed by the # character, for sections with dynamic data and these are replaced on runtime by the service triggering the message.

User message records

erDiagram

EmailQueue  {
 	bigint(8) ID "UniqueId"
	datetime(8) ToSend "Datetime when the e-mail was added to the queue for sending"
	datetime(8) Sent "Datetime when the e-mail was sent via roche mail server"
	nvarchar Email "Main recipient"
	nvarchar CC "Mail addresses, to which users the mail should be sent"
	nvarchar Subject "Subject of the mail"
	nvarchar Body "HTML body of the e-mail"
	nvarchar Exception "Error message occured via sending the mail"
}

Records classification and audit trail