Events & Messaging

• Introduction
• Batch Subscription Service
  • Messages
• Event Bus
  • Dashboard
  • Detection of removed items
  • Site events

Introduction

The following state diagram illustrates the state of a batch and the omitted events when the change of a state happens.

stateDiagram-v2
    [*] --> BatchActive: BatchStart
    BatchActive --> BatchActive: BatchValueChanged
    BatchActive --> BatchRelease: BatchEnd
    BatchRelease --> HistoricalBatch: Reprocessing
    [*] --> HistoricalBatch: Backfilling
    HistoricalBatch --> [*]

Batch Subscription Service

DCP Frameworks provides a shared service to check for currently active batches/events. Checks are performed using a fixed interval. The interval is configured in the web.config (CheckActiveBatches), the default value is set to 20s. State between the different execution cycles is maintained in the database. Therefore the table BatchIntermediated is used.

The service logic of active batches is illustrated in the figure below:

stateDiagram-v2
    [*] --> FindResources: 
    FindResources --> CollectSignups: Check ModuleConfigurationItems for resources with Type 1
    CollectSignups --> GetActiveEventFrames: Get Devices needed for the check from the resources
    DataSource --> GetActiveEventFrames
    GetActiveEventFrames --> DataSource
    GetActiveEventFrames --> FormatMessages: SignUps are grouped by the datasource Id and 1 request per datasource is performed
    GetEventFrames --> FormatMessages: Read snapshot of active eventframes from BatchIntermediate table
    FormatMessages --> DCPActiveBatchExchange: Batch start
    FormatMessages --> DCPActiveBatchExchange: Batch end
    FormatMessages --> DCPActiveBatchExchange: Batch value changes
 

On some transitions events are emitted - where modules can subscribe to. The event subscription uses topic based routing using the active batch exchanger.

Cache records

erDiagram

BatchIntermediate  {
 	int(4) ID "UniqueId"
	nvarchar BatchID "BatchId of the currently active eventframe"
	nvarchar(100) BatchStartTime "Start time of the active eventframe"
	nvarchar JBatchValues "Eventframe attributes, as JSON object"
	nvarchar(100) EventFrameId "UniqueId of the active eventframe returned by the datasource"
	nvarchar DeviceWebId "Relation of the eventframe to the asset/device on the datasource"
	bit(1) BadValue "Flag, if any eventframe attributes are marked as bad"
	int(4) UpdateAttems "Nbr of attemps made by the service to fix/update bad values"
	bit(1) IsNotifiedSend "Flag if the notification for unsucessful update of bad values has been sent"
	bit(1) IsNewSignUpCreated "Flag if the value was added within the latest cycle"
	int(4) SiteID "Relation to the site, to enable site depended computation offloading"
}

Records classification and audit trail

Messages

The following message formats are implemented:

Batch Start

{
  "DeviceWebId": "I1EmRuYiTbprtUev-SErXP9OeQ2WDNqNJU6xGQANxxlqOCiA",
  "SiteId": 3,
  "EventFrameId": "afa21edb-897f-11ec-90fd-005056a7fed6",
  "BatchStartTime": "2022-02-09T08:10:00Z",
  "BatchId": "1",
  "BatchValues": "[{\"Name\":\"Cond\",\"Value\":{\"Value\":\"Test\",\"Good\":true}},{\"Name\":\"BatchID\",\"Value\":{\"Value\":\"1\",\"Good\":true}}]",
  "BadValueDetected": false,
  "BatchEventType": 0,
  "Headers": "MVDA"
}

Batch End

{
    "DeviceWebId": "I1EmRuYiTbprtUev-SErXP9OeQ2WDNqNJU6xGQANxxlqOCiA",
    "SiteId": 3,
    "EventFrameId": "480b5be1-897e-11ec-90fd-005056a7fed6",
    "BatchStartTime": "2022-02-09T08:00:00Z",
    "BatchId": "1",
    "BatchValues": "[{\"Name\":\"Cond\",\"Value\":{\"Value\":\"Test\",\"Good\":true}},{\"Name\":\"BatchID\",\"Value\":{\"Value\":\"1\",\"Good\":true}}]",
    "BadValueDetected": false,
    "BatchEventType": 2,
    "Headers": "MVDA"
}

Batch Value Change

To avoid racing/timing problems when values are written to the EventFrame at the datasource level, DCP implements a logic for fixing bad/late-arriving values. Batch context information is saved into the database as JSON objects. Prior to storage, it is checked if the batch contains bad values. If Bad values are detected, DCP tries to update these values for the next 10 cycles of the checkActiveBatch execution. If the bad values could be fixed, a BatchValueChanges message is omitted to the message queue. This has the following format:

{
    "DeviceWebId": "I1EmRuYiTbprtUev-SErXP9OeQ2WDNqNJU6xGQANxxlqOCiA",
    "SiteId": 3,
    "EventFrameId": "480b5be1-897e-11ec-90fd-005056a7fed6",
    "BatchStartTime": "2022-02-09T08:00:00Z",
    "BatchEndTime": "2022-02-09T08:09:00Z",
    "BatchId": "1",
    "BatchValues": "[{\"Name\":\"Cond\",\"Value\":{\"Value\":\"Test\",\"Good\":true}},{\"Name\":\"BatchID\",\"Value\":{\"Value\":\"1\",\"Good\":true}}]",
    "BadValueDetected": false,
    "BatchEventType": 1,
    "Headers": "MVDA"
}

Event Bus

In DCP, the event bus is an asynchronous messaging service that facilitates cross-module communication. It allows messages to be published by one module and consumed by another, without the need for tight coupling between the modules. This design promotes a decoupled architecture, enabling modules to interact and share information efficiently while maintaining their independence. By using the event bus, DCP can achieve a scalable and flexible system where modules can be developed, updated, and maintained independently of each other.

The section below gives an overview of events and messages used in cross module communication:

Dashboard

• Deleting a dashboard tab (see: Dashboard import/Export)
• Updating the layout (see: Dashboard import/Export)
• Importing a dashboard (see: Dashboard update layout)

Detection of removed items

• Detection of a deleted/removed device on the datasource layer
• Detection of a deleted/removed sensor on the datasource layer

Site events

• When the site is deleted a message is distributed to the module services in order to remove corresponding values.