About the Enterprise Data Platform¶
Helping you to optimize the total cost of ownership for your data assets on EnOS, the EnOS Enterprise Data Platform has the following benefits:
Strong data analytics capability.
High data development efficiency and a lower barrier for stream data processing development.
Optimized data storage cost due to a more flexible configuration.
Effective data retention and access based on domain expertise accumulation.
Quick and easy data query for data insights.
Cleanup of historical data to release storage resource.
Multiple data quality measuring dimensions.
Metadata exploring of models and measurement points.
Subscription to the real-time data, alert data, and event data of assets.
The major components and architecture of EnOS Enterprise Data Platform is shown in the figure below.
Major Functions¶
EnOS Enterprise Data Platform provides the following major functions:
Data Asset Catalog Service¶
EnOS enables you to search the metadata of models and measurement points, including the basic information, stream processing lineage, and storage configuration of measurement points, and all the instances that are associated with the model to which a measurement point belongs. For more information, see Data Asset Catalog。
Data Federation Service¶
ENOS provides data reading and writing services for application developers and data analysts from and to multi-source heterogeneous data storage systems. You can read data from and write data to heterogeneous data sources through unified SQL statements. For more information, see Data Federation。
Time Series Data Management¶
The EnOS Time Series Database (TSDB) provides efficient and stable storage and management for time series data that is ingested from devices or generated by the stream processing engine by having a variety of data storage options based on your data storage and reading requirements. Data is stored by categories (data type and storage time), thus reducing data storage costs and enhancing data reading efficiency.
With the Data Insights feature, you can view the latest data ingested from your devices and also generate a chart for time series data that is stored in TSDB by selecting a specific time range and device measurement points, which improves the efficiency of data analysis.
With the Data Deletion feature, you can delete the historical data (wrong data, informal data, or business data that is no longer needed) stored in TSDB to release storage resource and save data storage costs. For more information, see Managing Time Series Data.
Note: The data storage policy must be configured before the devices start uploading data to EnOS Cloud. Otherwise, the uploaded device data will not be stored in EnOS TSDB.
Stream Processing Service¶
The EnOS Stream Processing Service provides the following to ensure the low-latency of stream data transmission and processing and high data analytics accuracy.
High throughput and low latency stream processing.
Commonly used stream processing algorithms in the IoT field.
Visualized template-based configurations to help you quickly perform your stream data processing jobs.
A set of underlying packaged StreamSets operators for developers to develop customized stream data processing jobs to meet the requirements of complex business scenarios.
For more information, see the overview of the Stream Processing Service.
Data Subscription Service¶
The data subscription service improves the API calling efficiency of applications with active data push, which supports subscription to various data types such as real-time asset data, asset alert data, and event data. You can use the provided Java SDK to retrieve the subscribed data. For more information, see Data Subscription Overview.
Data Archiving Service¶
EnOS supports archiving and storing data from the real-time message channel, data from the offline message channels, real-time alert records, and data stored in TSDB. It provides flexible storage configurations to reduce your data storage costs. For more information, see Data Archiving Overview.
Data Synchronization Service¶
EnOS supports synchronizing data between extensive heterogeneous data sources, such as synchronizing structured data from external source databases to the Hive database in EnOS and from the EnOS Hive database to external target databases. For more information, see Data Synchronization Overview.
Batch Processing Service¶
EnOS provides a GUI-based Dataflow Service that enables batch data processing based on scheduled workflows. The batch data processing operations support various scenarios from simple data transformation to complete ETL (extract-transform-load). The Batch Processing Service also provides common libraries out-of-the-box for most frequently used data processing operations. For more information, see Batch Processing Overview.
Data Quality Service¶
EnOS supports multi-dimensional data quality assessment. The StreamSets operators provided by the stream processing service can tag the quality of the streaming data, and generate data quality reports where you can check the completeness, accuracy, and timeliness of the data. For more information, see Data Quality Overview.
Data Service APIs¶
EnOS Enterprise Data Platform provides Data Service APIs for retrieving the stored data in EnOS TSDB. For detailed information about Date Service APIs, go to EnOS Management Console > EnOS APIs.