Unit 5. Run the Pipeline and View Pipeline Running Results


Run the Pipeline


In this tutorial, you need to run the pipeline twice to perform training tasks and prediction tasks. To run the pipeline:

  1. After saving the pipeline, select the Run icon in the functional operation area.

  2. Enter training as tasktype, and select OK to start training tasks.

  3. After the model training completes, select the Run icon and change the tasktype value to prediction.

  4. Enter service or file as predictiontype to specify the prediction type.

    • If you enter service as predictiontype, the pipeline will make predictions based on published model services.

    • If you enter file as predictiontype, the pipeline will make predictions based on Mlflow model files.

  5. Specify the value of HDFS_source and hive_source to save the prediction results to HDFS and EnOS Hive.

  6. Select OK to start prediction tasks.

View Pipeline Running Status and Prediction Results


During the pipeline running, you can view the pipeline running status by the following steps:

  1. Select Run Instances to view the instance list.

  2. Select the instance name of prediction tasks.

  3. View the running results on DAG Graph, Detail, and Gantt tabs.

    • On the DAG Graph tab, you can view the running results of running instance, including the running progress, input and output parameters, current running logs, and Pod information of operators.

    • On the Detail tab, you can view the details, running parameters, and advanced configuration of the running instance.

    • On the Gantt tab, you can view the running status distribution diagram of each operator and the running progress.

  4. After the pipeline is run successfully, you can view the operator outputs under the DAG Graph tab:

    • If the predictiontype value is service, select the Write results operator to view the input/output parameter information.

    • If the predictiontype value is file, select the Write results 2 operator to view the input/output parameter information.

  5. To view the prediction results:

    • Select Create DataSet icon create_dataset to create a dataset based on the output and analyze the data. For more information, see Dataset Management.

    • Select Copy icon copy_icon, and then open a Notebook instance in AI Lab, open a new document to run the pasted code.

    • If the prediction results are saved to EnOS Hive, you can visualize the prediction results using EnOS Digital Twin Visualization (DTV).

(Optional) Visualize the Prediction Results in DTV


The EnOS DTV service provides enterprises data visualization solutions. You can use DTV to visualize the prediction results if:

  • You are able to access the Data Federation module in the EnOS Management Console and the DTV in the EnOS Application Portal.

  • Your OU has requested for EnOS Hive and Data Federation resources.


For more information, see Digital Twin Visualization.

Create a Data Federation Channel


Create a Data Federation channel to send the pipeline data to DTV:

  1. Log in to the EnOS Management Console and select EnOS Data Management - Data Governance > Data Federation > Federation Channels.

  2. Select New Channel, enter wind_demo as Name, then provide the following information in the popup window:


    Attribute

    Value

    Data Source > Add Data Source > Type

    HIVE (EnOS)

    Data Source > Add Data Source > Data Source

    HIVE (EnOS)

    Data Source > Add Data Source > Alias

    hive_enos

    Data Source > Add Data Source > Execution Queue

    root.eaptest01


  3. Select Save and Next.

  4. In the application list, select the Authorize/Revoke toggle button on the right side of the DTViz application to authorize permission for this application.

  5. Select Finish and note the channel ID.

Connect the Data with DTV


Create a DTV Data Source to receive the pipeline data:

  1. Log in to DTV in the EnOS Application Portal, select Dashboards > Data Source on the left navigation pane.

  2. On the External Data Sources tab, select New Data Source and provide the following information in the popup window:


    Attribute

    Value

    Type

    Data Federation

    Name

    wind_demo

    ChannelId

    ID of the data federation channel just created


  3. Select OK.

Create a Project and a Page in DTV


Create a new project and a new page in DTV:

  1. Select Dashboard on the left navigation pane, and the project page opens.

  2. select New Project and enter Power Forecasting as Name in the popup window.

  3. Select OK, and the project creates.

  4. Select New Page and provide the following information.

    • Name: Wind

    • Project: Power Forecasting

  5. Select OK, and the page editor opens.

Create a Widget in DTV


Create a DTV widget to visualize the prediction results:

  1. On the page editor, select Add icon add_button on the toolbar to open the dropdown menu and select Widget (New).

  2. Select the Advanced Mode toggle button on the right side of the top toolbar to enable more widget editing functions.

  3. From the Chart Type section, select Mix Chart mix_chart.

  4. Enter Wind Power Forecast as Title in the Basic Information section.

  5. In the Data Fields section, provide the following information:


    Attribute

    Value

    Data Source

    wind_demo

    Category

    Power Prediction

    Aggregation

    Raw Data

    Dimension

    x_basic_time

    Measurement

    r_speed, f_power, r_power, f_speed


  6. In the Measurement Style section, provide the following information:


    Attribute

    Dual Y-axis

    Chart Type

    r_speed

    Enable

    Line

    f_speed

    Enable

    Line

    r_power

    Disable

    Area

    f_power

    Disable

    Area


  7. Select Save. You can now view the predicted values and actual values in the chart.


An example of the Wind Power Forecast widget:


../_images/wind_power_forecast.png