Data Gap Backfill

EnOS Edge supports Data Gap Backfill. If the connection between EnOS Edge and EnOS cloud is interrupted, the Edge can continue to collect data from sub-devices and cache it locally. The data cached locally will be transmitted again once the network connection is restored.The data processing method is shown in the following figure.

../_images/backfill.png

Application

It supports data breakpoint continuous transmission between the following systems under the scenario of network disconnection or weak network:

  • EnOS Edge - EnOS Edge

  • Edge SCADA - EnOS Edge

  • Third Party SCADA - EnOS Edge

  • EnOS Edge - EnOS Cloud

  • EnOS Edge - Third Party SCADA

Features

EnOS Edge Data Gap Backfill has the following features:

  • Set the threshold to judge current data is real-time or historical

  • Provide measurement tags to process historical data ( discard, re-filled through real-time topic or re-filled through historical topic)

  • Provide data compression function to restore more data

  • Support data feed strategy to insure real-time data priority

Configuration Items

The configuration items are used to adapt to many scenarios. The current default value can meet the needs of the most common scenarios. Also, you can change the following default value of the common items for specific scenarios. If you need to change other configuration items, contact the staff of EnOS Edge first.

Item

Description

Default Value

config-manager.box.data.history.switch

Whether to enable history channel (enabled by default after v2.3.0)

True

config-manger.box.break.point.delay.time

The delay time to judge whether the data in the real-time stream has entered the historical queue

120 Minutes (Data that delay over 120 minutes are regarded as historical data)

config-manager.box.break.point.realtime.data.topic.capacity

The capacity of real-time Topic is used to set the storage capacity

100 GB

config-manager.box.break.point.history.data.topic.capacity

The capacity of historical Topic is used to set the storage capacity

100 GB

config-manager.box.break.point.topic.timeout

The maximum cache time (historical and real-time data retention time when resuming a breakpoint)is used to set the storage capacity

720 hours

config-manger.box.realtime.data.max.lag

The backlog message limit of real-time Topics. When the number of messages in the real-time topic exceeds 1000, the historical topic will be suspended and only the real-time topic data will be uploaded.

1000

Note

It is recommended to set the storage capacity according to your actual situations. The upper limit of the actual storage Topic capacity takes the smallest value among the Topic capacity, disk size, and the maximum cache time.