FAQs¶
Q: Do data archiving jobs support both automatic and manual modes?¶
A: The running of data archiving jobs is rule-driven. You need to configure data archiving jobs based on your business needs (such as where to store the archived data, which data to archive, and the archiving cycle). When a data archiving job is started and running, data will be archived according to the configuration without human intervention.
Currently, data archiving supports Real-Time and Offline job types. For real-time job type, the data archiving job will keep running. Once data is generated from the data source, the job will archive the data according to the configuration automatically. For offline job type, the job will run only once. After all the data specified in the configuration is archived, the job will stop running.
Q: What will be impacted if the configuration of a running data archiving job is modified?¶
A: After the data archiving job configuration is modified and submitted, the updated configuration will take effect immediately. The data that has been archived will not be impacted. For example, if the storage path of archived data is changed from /tds/ods/alarm1/ to /tds/ods/alarm2/, the new storage path will take effect immediately after the change is submitted. After about 1-2 minutes, the archived data will be stored in the alarm2 directory. The archived data that has been stored in the alarm1 directory will not be impacted.
Q: How to query the data that has been archived in the target storage?¶
A: The Data Archiving service enables archiving data from the data sources to the target storage. It is a set of archiving job configuration and management tools, but it does not provide the management of the target storage systems, nor the query ability of archived data. You need to use the corresponding management tools of the target storage systems for data query. For example:
- If the target storage is EnOS HDFS, you can use the Jupyter Notebook that is provided AI Studio > AI Lab product to query data stored in HDFS. For information, see Managing Notebook Instance.
- If the target storage is Azure Blob, you can use the client tools provided by Azure platform to query the data stored in Blob Storage.
Q: When the data archiving job is restarted after running failure, will the job re-archive the data at the moment when the job fails?¶
A: For the following situations:
- For real-time data archiving, when the failed job is restarted, it will re-archive all the failed data in the last 3 days automatically. If the job failed for more than 3 days, it can process data in the latest 3 days only. Therefore, when the data archiving job failure triggers the alert notification through SMS or email, the alert receiver must take action in time to avoid data loss.
- For offline data archiving, when the failed job is restarted, it will re-archive all the data again.