Archive Service


The light-weight archiving service enables you to archive large volumes of infrequently used data to a distributed file system asynchronously.


The types of data that can be archived include the following.

  • Raw Data:

    • Raw data received through analog input (AI)

    • Raw data received through digital input (DI)

    • Raw data received through pulse input

    • Generic raw data

  • Alerts

  • Custom data:

    • 5-minute telemetry data archiving

    • 10-minute telemetry data archiving

Raw Data Archiving

Archiving Configuration

Archiving Service Parameter

Parameter Value

User

General

Archiving Start Time

Specified when the archiving is created and cannot be changed later. Starts from the day before the archiving is created by default.

Period

1 day

Archiving Model and Measurement Point Selection

You can select the model and measurement points to be archived.

Path

The format of the path is: data_type/date/model_ID. For example, AI/20200202/EnOS_Solar_Inverter

Archived File Name

The file is named after the asset ID.

Encoding

UTF-8

Filename Extension

.csv

Format

As described below.

Column Separator

Specified when the archiving is created and cannot be changed later. The default separator is the comma.

File Size

No upper limit.

Compressed / Not Compressed

Specified when the archiving is created and cannot be changed later. Compressed by default.

Compression Algorighm

GZIP

Merge / Do Not Merge Files of the Same Type

Does not merge by default.

Format of AI, PI, Generic Archived Files

An example to illustrate the format of AI, PI, Generic archived files is as per the below, where the user decides to archive the data of the following measurement points.

Measurement Point Information

Measurement Point ID

Name (English)

Name (Chinese)

WTUR.GenActivePW

Reactive Power Production

有功功率

WTUR.WindSpeed

Wind Speed

风速


In an archived file, the headers take up the first 3 rows as per the following.

Measurement Point Information

Asset ID

Asset Name

Data Time

Data Time UTC

WTUR.GenActivePW

WTUR.WindSpeed

Asset ID

Asset Name

Data Time

Data Time UTC

Reactive Power Production

Wind Speed

资产 ID

资产名称

时间

UTC时间

有功功率

风速

WindturbineA

Wind Turbine A in China

2020-01-01 00:00:00

2020-01-01 00:00:00

123.123

4.444


In this sample, the headers are displayed using the ID, English name, and Chinese name. The first 4 columns are fixed and the measurement point data starts from the 5th column.


Measurement points are displayed in an ascending order based on their priority. You can configure the priority of a measurement point by specifying the priority in a tag. Measurement points without priority tags are placed last by default.

Format of DI Archived Files

An example to illustrate the format of DI archived files is as per the below, where the user decides to archive the data of the following measurement points.

Measurement Point Information

Measurement Point ID

Name (English)

Name (Chinese)

WTUR.STA1

Status #1

状态1

WTUR.STA2

Status #2

状态2


In an archived file, the headers take up the first 3 rows as per the following.

Measurement Point Information

Asset ID

Asset Name

Data Time

Data Time UTC

Point ID

Point Value

Asset ID

Asset Name

Data Time

Data Time UTC

Point ID

Point Value

资产 ID

资产名称

时间

UTC时间

测点ID

数值

WindturbineA

Windturbine A in China

2020-01-01 00:00:00

2020-01-01 00:00:00

WTUR.STA1

1

WindturbineA

Windturbine A in China

2020-01-01 00:00:00

2020-01-01 00:00:00

WTUR.STA2

0

WindturbineB

Windturbine B in China

2020-01-01 00:00:00

2020-01-01 00:00:00

WTUR.STA1

0

WindturbineB

Windturbine B in China

2020-01-01 00:00:00

2020-01-01 00:00:00

WTUR.STA2

1


In this sample, the headers are displayed using the ID, English name, and Chinese name. The measurement point data starts from the 5th column.


The measurement points are displayed in the same manner as the AI, PI, and Generic data, i.e., in ascending order based on their priority, etc.

Alert Archiving

Archiving Configuration

Archiving Service Parameter

Parameter Value

User

General

Archiving Start Time

Specified when the archiving is created and cannot be changed later. Starts from the day before the archiving is created by default.

Period

1 day

Archiving Model and Measurement Point Selection

All models and measurement points.

Path

The format of the path is: data_type/date/model_ID. For example, AI/20190909, DI/20190909.

Archived File Name

The file is named after the asset ID.

Encoding

UTF-8

Filename Extension

.csv

Format

As described below.

Column Separator

Specified when the archiving is created and cannot be changed later. The default separator is the comma.

File Size

No upper limit.

Compressed / Not Compressed

Specified when the archiving is created and cannot be changed later. Compressed by default.

Compression Algorighm

GZIP

Merge / Do Not Merge Files of the Same Type

Does not merge by default.

File Cleanup Policy

Specified when the archiving is created and cannot be changed later. 1% by default.

Format of Archived Files

EnOS Edge uses the same method to archive historic alerts as EnOS Cloud.

Alert Archiving File Format

File Archived

Raw Data

Note

eventId

eventId

The alert ID that is unique within an OU and cannot be changed.

orgId

orgId

The OU ID to which the asset belongs.

assetId

deviceId

The asset ID.

modelId

modelId

The model ID to which the asset belongs.

modelIdPath

modelIdPath

The path of this model.

measurepointId

pointId

The measurement point ID.

value

value

The measurement point value.

occurTime

time

The time when the alert is triggered in UTC time.

localOccurTime

localTime

The time when the alert is triggered in local time. A 13-digit timestamp is stored in Elasticsearch. The API provided by the alert service converts the timestamp into yyyy-mm-dd hh:mm:ss, rounded up to seconds.

recoverTime

recoverTime

The time when the alert is dismissed in UTC time.

recoverLocalTime

recoverLocalTime

The time when the alert is dismissed in local time. A 13-digit timestamp is stored in Elasticsearch. The API provided by the alert service converts the timestamp into yyyy-mm-dd hh:mm:ss, rounded up to seconds.

recoverReason

recoverReason

The reason why the alert is dismissed.

createTime

createTime

The time when the alert is archived in UTC time.

updateTime

updateTime

The time when the alert record is updated in UTC time.

severityId

severityId

The user-defined alert severity ID.

severityDesc

severityDesc

The description for the alert severity.

typeId

typeBid

The user-defined alert type ID.

typeDesc

typeDesc

The description for alert type.

subTypeId

subTypeBid

The user-defined alert sub-type ID.

subTypeDesc

subTypeDesc

The description for the alert sub-type.

contentId

contentBid

The user-defined alert content ID.

contentDesc

contentDesc

The description for the alert content.

eventType

eventType

The event type.

  • 0: Dismissed by the system

  • 1: Triggered by the system

  • 2: Dismissed manually

  • 3: Inserted manually

tag

tag

The alert tag. The API provided by the alert service will convert the tag into standard JSON object [key1:v1,key2:v2].

ruleDesc

ruleDesc

The description for the alert rule.

assetPaths

assetPath

The scope where the alert rule takes effect and the path of the device where the alert is triggered is returned. The format is [“treeId1:/assetId1/assetId2/assetIdx”, “treeId2:/assetId3/assetIdx”].

maskedBy

maskedBy

If this alert is from a sub-device, the root alert that masks this alert is returned. If this alert has multiple root alerts, all the root alerts that can mask this alert is returned and an eventId is generated. The format is [“treeId1:eventId1”, “treeId1:eventId2”].

Note

  • The timestamps for localOccurTime and recoverLocalTime are stored in MySQL. Alert service APIs will convert the timestamp to datetime-type data and archive the converted data. The format of the datetime data is yyyy-mm-dd hh:mm:ss.

  • Structs, such as {} or [], will be stored as string without braces {} or brackets [].

  • If no data is archived during the specified period, a file that contains only headers will still be generated.

  • The compression method is GZIP by default with the filename extension .gz.

Archiving 5-minute and 10-minute Telemetry

Archiving Configuration

Archiving Service Parameter

Parameter Value

User

General

Archiving Start Time

Specified when the archiving is created and cannot be changed later. Starts from the day before the archiving is created by default.

Period

1 day

Archiving Model and Measurement Point Selection

All models and measurement points.

Path

The format of the path is: data_type/date/. For example, AI/20190909, DI/20190909.

Archived File Name

The file is named after the asset ID.

Encoding

UTF-8

Filename Extension

.csv

Format

As described below.

Column Separator

Specified when the archiving is created and cannot be changed later. The default separator is the comma.

File Size

No upper limit

Compressed / Not Compressed

Specified when the archiving is created and cannot be changed later. Compressed by default.

Compression Algorithm

GZIP

Merge / Do Not Merge Files of the Same Type

Does not merge by default.

File Cleanup Policy

Specified when the archiving is created and cannot be changed later. 1% by default.

Format of Archived Files

An example to illustrate the format of the archived file is per the below.

Measurement Point Information

Asset ID

Asset Name

Data Time

Data Time UTC

WTUR.GenActivePW_AVG_5m

WTUR.WindSpeed_AVG_5m

Asset ID

Asset Name

Data Time

Data Time UTC

Reactive Power Production(avg 5m)

WindSpeed(avg 5m)

资产 ID

资产名称

时间

UTC时间

有功功率五分钟平均值

风速五分钟平均值

windTurbineC

windTurbine C in China

2020-01-01 00:00:00

2020-01-01 00:00:00

123.123

4.444


In this sample, the headers are displayed using the ID, English name, and Chinese name. The measurement point data starts from the 5th column.


The measurement points are displayed in the same manner as AI, PI, and Generic data, i.e., in ascending order based on their priority, etc.

Default File Path

Data Type

Path

AI Raw Data

/EDGE/ARCHIVE/ORIGINAL_AI/

DI Raw Data

/EDGE/ARCHIVE/ORIGINAL_DI/

PI Raw Data

/EDGE/ARCHIVE/ORIGINAL_PI/

GENERAL Raw Data

/EDGE/ARCHIVE/ORIGINAL_GENERIC/

AI 5-minute Data

/EDGE/ARCHIVE/FIVE_MIN_AI/

AI 10-minute Data

/EDGE/ARCHIVE/TEN_MIN_AI/

ALert

/EDGE/ARCHIVE/ALARM/