Data Backfill¶
Edge scenarios often face disconnection and weak network problems, which will cause communication obstruction, data backlog or loss. EnOS Edge supports data backfill in various scenarios, that is, Edge caches data in a network disconnection and continues to transmit cached historical data after communication resumes. Taking the data backfill between EnOS Edge and EnOS cloud as an example, data backfill can avoid data loss caused by communication failures, and ensure that applications in EnOS cloud can calculate and analyze historical data over a period of time. The data processing method is shown in the following figure.
Application¶
It supports data backfill between the following systems:
- EnOS Edge - EnOS Cloud
- EnOS Edge - EnOS Edge
- Edge SCADA - EnOS Edge
- Third Party SCADA - EnOS Edge
- EnOS Edge - Third Party SCADA
Features¶
EnOS Edge data backfill has the following features:
- Provide measurement tags to process historical data ( discard, re-filled through real-time topic or re-filled through historical topic)
- Provide data compression function to restore more data
- Support data feed strategy to insure real-time data priority
Configuration Items¶
The default values of configuration items for data backfill can meet the requirements of common scenarios. If there are additional usage requirements, the following items can be changed during the deployment process.
| Item | Description | Default Value |
|---|---|---|
| config-manager.box.data.history.switch | Whether to enable history channel (enabled by default after v2.3.0) | True |
| config-manager.box.break.point.realtime.data.topic.capacity | The capacity of real-time Topic is used to set the storage capacity | 100 GB |
| config-manager.box.break.point.history.data.topic.capacity | The capacity of historical Topic is used to set the storage capacity | 100 GB |
| config-manager.box.break.point.topic.timeout | The maximum cache time (historical and real-time data retention time when resuming a breakpoint）is used to set the storage capacity | 720 hours |
Note
It is recommended to set the storage capacity according to your actual situations. The upper limit of the actual storage Topic capacity takes the smallest value among the Topic capacity, disk size, and the maximum cache time.