Archiving Logs


Using the Log Archiving feature provided by Cluster Management, the logs collected by the cluster can be quickly archived in a specific format to a specified storage, and when needed, the archived logs can be restored to the environment for query and analysis like normal logs.

Prerequisites

  • User needs to have access to cluster management (Administrator, Cluster Administrator, or Namespace Administrator)
  • Clusterand namespace names have been selected
  • Storage required for log archiving has been initialized

Create a Database

Create a library by following these steps:

  1. From the left navigation bar, select Log > Log Archiving to view all archived log files.

  2. Under the Repository tab, click Create Repository, enter a Repository name, and select a Repository Type.

  3. According to the selected Repository Type, complete the configuration of the address information of the repository.

    ../../../_images/creating_log_repo.png

  4. Click Confirm to complete the repository creation.

Creating Archiving Policy

Once the repository is created, you can try to create an archiving policy. To create an archiving policy, proceed as follows.

  1. Under the Archiving Policy tab, click New Policy and complete the following archiving policy configuration in a new window.

    • Policy Name: Enter the name of the archiving policy
    • File Prefix: Enter the prefix of the generated archive file
    • Repository: select the name of the created repository
    • Index: log index, by default all indexes in the cluster will be archived; if you turn off all indexes, you can select the indexes that need to be archived. Index type refer to the following description.
    • Storage Time: how long the archived files are kept in the system
    • Frequency: select the frequency of running log archiving tasks
    ../../../_images/log_archiving_policy.png

  2. Click Confirm to submit the archiving policy. The system will trigger the archiving task according to the policy.

Index Type Description

Index Types Description
kublog-app-* Application type logs, which can be queried in the Log after restoring the archived content.
kublog-serverless-* Serverless type logs, which can be queried in Log after restoring the archived content.
kublog-spark-* The archive is not available in Log , but can be queried through the interface.
kublog-kube-eventer-* The archive is not available in Log , but can be queried through the interface.
kublog-job-* The archive is not available in Log , but can be queried through the interface.
kublog-kube-system-* The archive is not available in Log , but can be queried through the interface.

Managing Archived Files

When both the repository and archive policies are in effect, the system will generate the archive files according to the rules defined in the policy. You can view the generated results in the archived file list.


../../../_images/archived_log_files.gif


Generated archive files contain log indexes at the archive time node, which you can manipulate, such as actively deleting or restoring the archive file before its retention time expires.

Note

Log archiving, restoring, and deleting operations will be executed serially in the background after triggering. If log archiving and restoration operations are performed at the same time, the performance of the log service will be affected, so please be careful.

Manage Archive Restoring

Switch to the Archive Restoring page to view the status of recovery results for archived files. You can filter by archive file name, repository name, time range and recovery status. You can also delete the recovered archives from the cluster.

../../../_images/archive_restoration.gif