About the Enterprise Data Platform¶
Helping you to optimize the total cost of ownership for your data assets on EnOS, the EnOS Enterprise Data Platform has the following benefits:
Strong data analytics capability.
High data development efficiency and a lower barrier for stream data processing development.
Optimized data storage cost due to a more flexible configuration.
Effective data retention and access based on domain expertise accumulation.
Quick and easy data query for data insights.
Cleanup of historical data to release storage resource.
Multiple data quality measuring dimensions.
Data exploring of models and measurement points.
Subscription to the real-time data, alert data, and event data of assets.
The major components and architecture of EnOS Enterprise Data Platform is shown in the figure below.
Major Functions¶
EnOS Enterprise Data Platform provides the following major functions:
Data Catalog Service¶
EnOS Data Catalog Service provides a unified management of massive amount of data from multiple sources and implements data modelling and data standards to realize efficient data discovery, unlocking data value and helping business users to understand enterprise data assets easily. For more information, see Data Catalog.
Stream Processing Service¶
The EnOS Stream Processing Service provides the following to ensure the low-latency of stream data transmission and processing and high data analytics accuracy.
High throughput and low latency stream processing.
Commonly used stream processing algorithms in the IoT field.
Visualized template-based configurations to help you quickly perform your stream data processing jobs.
A set of underlying packaged StreamSets operators for developers to develop customized stream data processing jobs to meet the requirements of complex business scenarios.
For more information, see the overview of the Stream Processing Service.
Time Series Data Management¶
The EnOS Time Series Database (TSDB) provides efficient and stable storage and management for time series data that is ingested from devices or generated by the stream processing engine by having a variety of data storage options based on your data storage and reading requirements. Data is stored by categories (data type and storage time), thus reducing data storage costs and enhancing data reading efficiency.
With the Data Insights feature, you can view the latest data ingested from your devices and also generate a chart for time series data that is stored in TSDB by selecting a specific time range and device measurement points, which improves the efficiency of data analysis.
With the Data Deletion feature, you can delete the historical data (wrong data, informal data, or business data that is no longer needed) stored in TSDB to release storage resource and save data storage costs. For more information, see Managing Time Series Data.
Note: The data storage policy must be configured before the devices start uploading data to EnOS Cloud. Otherwise, the uploaded device data will not be stored in EnOS TSDB.
Data Federation Service¶
ENOS provides data reading and writing services for application developers and data analysts from and to multi-source heterogeneous data storage systems. You can read data from and write data to heterogeneous data sources through unified SQL statements. For more information, see Data Federation.
Data Subscription Service¶
The data subscription service improves the API calling efficiency of applications with active data push, which supports subscription to various data types such as real-time asset data, asset alert data, and event data. You can use the provided Java SDK to retrieve the subscribed data. For more information, see Data Subscription Overview.
Data Archiving Service¶
EnOS supports archiving and storing data from the real-time message channel, data from the offline message channels, real-time alert records, and data stored in TSDB. It provides flexible storage configurations to reduce your data storage costs. For more information, see Data Archiving Overview.
Data Synchronization Service¶
EnOS supports synchronizing data between extensive heterogeneous data sources, such as synchronizing structured data from external source databases to the Hive database in EnOS and from the EnOS Hive database to external target databases. For more information, see Data Synchronization Overview.
Batch Processing Service¶
EnOS provides a GUI-based Dataflow Service that enables batch data processing based on scheduled workflows. The batch data processing operations support various scenarios from simple data transformation to complete ETL (extract-transform-load). The Batch Processing Service also provides common libraries out-of-the-box for most frequently used data processing operations. For more information, see Batch Processing Overview.