Late Point Tagger*¶
This stage supports the recognizing of late data and the tagging of late data points for generating data quality reports. The specific logic is as follows:
- Use the
timestamp
field as the standard to filter late records. If thetimestamp
of a record is “later” than that of the latest record, the record will be recognized as late data. - The judging logic applies to records of the same point and same device.
- For the first arriving record of a new point and new device, it will be recognized as “not late”. Its timestamp will be used to judge records arriving after it.
- This stage supports streaming lineage.
- This stage cannot guarantee idempotence of the calculation results due to failure retries caused by any reasons, such as cluster node exceptions.
- Incompatible with old data formats, i.e. cannot filter data by
ModelId::PointId
.
Configuration¶
The configuration tabs for this stage are General, Basic, Input/Output, and CacheConfig.
General¶
Name | Required? | Description |
---|---|---|
Name | Yes | The name of the stage. |
Description | No | The description of the stage. |
Stage Library | Yes | The streaming operator library to which the stage belongs. |
Required Fields | No | The fields that the data records must contain. If the specified fields are not included, the record will be filtered out. |
Preconditions | No | The conditions that must be satisfied by the data records. Records that do not meet the conditions will be filtered out. For example, ${record:value('/value') > 0} . For the syntax of EL expressions, see Expression Language. |
On Record Error | Yes | The processing method for error data.
|
Basic¶
Name | Required? | Description |
---|---|---|
Quality Filter | No | Filter the data according to the data quality. Only records that meet the quality conditions will be processed by this stage. |
Input/Output¶
Name | Required? | Description |
---|---|---|
Input Measurement | Yes | Data input point. The MeasurementId must be different between the input and output points of the same row. |
Output Measurement | Yes | Data output point. The MeasurementId must be different between the input and output points of the same row. |
CacheConfig¶
Name | Required? | Description |
---|---|---|
Cache Type | Yes | Select the storage type for cache data. Options are Redis and Local storage.
|