Main Functions¶

EnOS Archiving service supports archiving and storing real-time data, data stored in TSDB, data processed by the stream processing service, and other business data with low access frequency and large storage space. The data flow is shown in the following figure.

Archiving Real-time Data and Offline Data¶

The Data Archiving Service supports the archiving of both real-time data ingested from devices or generated by stream processing jobs of specified models and data integrated from the offline message channel.

Archiving Real-time Alert Records¶

The Data Archiving Service supports the archiving of specified fields of real-time alert records (both history and active alert records) of specified models.

Archiving Data Stored in TSDB¶

The Data Archiving Service also supports the archiving of history data that is stored in TSDB. The data archiving job for TSDB data is offline type and needs to be triggered manually.

Setting the Properties of Archived Files¶

Properties of archived files, including file type, encoding, column delimiter, compression, and size limit, can be set based on your data usage scenarios. In this way, the archived files will be ready for future development and analysis.

Customized Archiving Cycles¶

You can set customized data archiving cycles (1 hour, 12 hours, or 24 hours) based on the data amount and business requirements on data archiving efficiency. The longer the archiving cycle is, the more data can be processed in the cycle, and the number of small files caused by data latency can be significantly reduced.

Setting the Storage System and Path**¶

The archived files will be synchronized to the specified storage system (BLOB or HDFS) and stored in the configured path, thus achieving data backup.