Charting
Introduction
The DCP UI heavily depends on charts for presenting the data/analysis results. Charts are generated on the FE or within the calculation node. The decision where charts are generated can vary and depends on the following factors:
- sizes of messages to be transferred chart vs. raw data message size
- Availability of data (DCP DB, BE vs. calculation node)
- Amount of interaction with the chart
No matter of the layer the library plotlyJS is used for rendering/generating the chart object. Interactive charts generated on the calculation node are build using the plotly package and converted to a JSON string using a custom helper. This helper reduces message sizes, compared to the default conversion, significantly. For static charts e.g. inside reports, the ggplot2 library is used. Normally functions generating charts have two separated implementations, as the automatic conversions (ggplotly) does not allow enough control or increases message sizes drastically. With a format specifier, the interactive/static chart can be displayed. Seeing DCP as a data intensive application, charting with big amount of data points is getting a critical success factor for system performance.
Chart optimization techniques
The following techniques are implemented in order to optimize the chart rendering in the system.
WebGL vs svg
Browser based applications can render graphics using the SVG API which supports vector rendering, and the Canvas API which supports raster rendering, and can exploit GPU hardware acceleration via a browser technology known as WebGL. WebGL is a powerful technology for accelerating computation but comes with some strict limitations. DCP uses WebGL for rendering plots whenever possible.
Server side rendering
A known performance limitation of plotly is the handling of many traces (> 50) in the same chart. Even a relatively low number of total data points can result in very bad rendering performance if they are distributed across multiple traces. In cases this is DCP uses the concept of server side rendering. Instead of returning data to the UI and render the chart in the browser a static image generated and rendered on the calculation node is returned to the user and displayed. Chart interactions e.g. zooming/tooltips are implemented by reloading the image with adjusted zoom levels or based on an interaction with the calculation node.
Reducing the amount of data
The problem of data points is not unique to rendering, it applies as well to storage, etc. Therefore, many concepts for reducing the amount of data has been established. DCP is following the implementation performed in Aveeva PI where three methods for data retrieval are known:
- Interpolated data: Retrieves interpolated values over the specified time range at the specified sampling interval
- Recorded data: the compressed values are returned, if no values were recorded at this point (no or only insignificant changes) the response is empty.
- Plot data: returns data optimized for plotting. The number of intervals is set in DCP to be a forth of the available pixel for plotting. For each interval, the data available is examined and significant values are returned. Each interval can produce up to 5 values if they are unique, the first value in the interval, the last value, the highest value, the lowest value and at most one exceptional point (bad status or digital state)