Basic Concepts¶

Channel¶

The Data Federation service provides a unified data access layer that shields the differences of heterogeneous data sources. Users with different roles can efficiently access data through unified query methods such as standard SQL, JDBC, or OData, thus saving the costs of creating a centralized data warehouse and avoiding the workload and resource waste of massive data replication.

Channel Type¶

The Data Federation service supports 2 types of channel: the read channel and the write channel. The read channel supports data query from specified data source, and the write channel supports data writing to specified data source.

Read Channel

After configuring and starting the read channel, data from the associated data source can be read through APIs.
The data sources supported by the read channel include MySQL, Blob, HDFS(EnOS), HIVE(EnOS), KAFKA(EnOS), Redis, and S3.

Write Channel

After configuring and starting the write channel, data can be written to the associated data soruce through the SDK.
The data sources supported by the write channel include KAFKA(EnOS), MySQL, HDFS(EnOS), and Redis.

Channel Authorization¶

The data federation channel support authorization and access authentication functions. When creating a channel, you can give permissions to the designated service accounts to access the channel.

Single-Source Query¶

Data Federation supports querying data from Hive or MySQL data source with native SQL of the data source. When starting a read channel, you can specify whether to enable the Multiple-Source Query function. This function requires at least 1CU running resource.

Multiple-Source Query¶

Data Federation supports joint data query among multiple data sources with unified SQL statements. When starting a read channel, you can specify whether to enable the Multiple-Source Query function. This function requires at least 2CU running resource.