Model Operators


The model operators are used to implement various system functions in the AI Pipelines, such as new model creation (analogous to the model registration in the AI Hub), model version staging, model instance creation, single model version online deployment, and prediction based s well as system functions based on the deployed model services or model files. It helps to achieve a closed loop of business from model training to service deployment and prediction based on the trained model.


The AI Pipelines provides the following operators related to registering and deploying machine learning models:

  • Model (model registration)

  • Mlflow Model Version Register (model version staging)

  • Docker Model Version Register (model version staging)

  • S2i Model Version Register (model version staging)

  • Thirdparty Model Version Register (model version staging)

  • Latest Model Version

  • Mlflow Model Version Prediction (prediction based on the machine learning image)

  • Model Instance (creating a model deployment instance)

  • Model Test (model testing)

  • Single Model Deployment (single model version deployment)

  • Model Version Info (Getting Model Version Information in multi-deployment scenarios)

  • Model Deployment (multiple model version deployment)

  • Service Prediction (prediction based on the model deployment service)

  • External Mlflow Prediction (prediction with the model file recorded by Mlflow in an external environment)

  • Internal Mlflow Prediction (prediction with the model file recorded by Mlflow in an internal environment)

  • Model Monitor (model performance monitoring)

  • Model Log (Getting model logs by time or key words)

Model Operator


The Model operator is used to create a new model (similar to the model registration function in the AI Hub).

Input Parameters Description


Name

Required/Optional

Type

Description

category

Required

String

Model category; options:

  • Predictor

  • Combiner

  • Data Transformer

model_name

Required

String

Model name, where lower cases, numbers and dash are supported with the length limited within 32 characters.

alias

Optional

String

Alias of the model.

input_data_type

Required

String

Input data type; options: Text, Tabular or Image.

scope

Required

String

Model application scope; options: Private or Public.

technique

Required

String

Model technology attribution; options:

  • Classification

  • Regression

  • Clustering

  • Other

usecase

Required

String

Model application field; options:

  • Wind

  • Solar

  • Power Storage

  • Smart Factory

  • Smart City

  • Other

publisher

Required

String

Model creator name

input_format

Required

String

Input parameters of the model feature in JSON format. See input_format sample.

output_format

Required

String

Model target output in JSON format. See output_format sample.

interface

Required

String

API type; options: REST or GRPC (that is, the calling method of model service API).

error_on_exist

Optional

Boolean

Specify whether to skip without reporting an error when the model name exists. If false is selected, no error will be reported; if `true`is selected and the name of the model to be created already exists, an error will be reported directly.

input_format sample


[
  {
       "name":"X-basic.hour",
       "dtype":"int",
       "ftype":"continuous",
       "range": [
           0,
           23
       ],
       "annotations":"",
       "repeat":null,
       "defaultValue":10
   },
    {
       "name":"X-basic.horizon",
       "dtype":"int",
       "ftype":"continuous",
       "range": [
           0,
           50
       ],
       "annotations":"",
       "repeat":null,
       "defaultValue":8
   }
]

output_format sample


[
  {
    "name": "test",
    "dtype": "float",
    "ftype": "continuous",
    "range": [
      1,
      11
    ],
    "annotations": null,
    "repeat": 4,
    "defaultValue": 1
  }
]

Output Parameters Description


Name

Type

Description

model_name_output

String

Output the created model name, which is used as the input of Mlflow Model Version Register and other operators.

Mlflow Model Version Register Operator


The Mlflow Model Version Register operator is used to create the model version file and stage the model version by means of MLflow import.

Input Parameters Description


Name

Required/Optional

Type

Description

input_data

Required

String

Model version parameter input. See the input_data sample.

alias

Optional

String

Alias of the model.

version_rule

Required

String

Model version naming rule, where the time basis is taken by default to name the model version according to the current timestamp.

annotation

Optional

String

Model version description information.

architecture

Required

String

Basic hardware for running model version; options: x86 or arm.

coprocessor

Optional

String

Select the coprocessor for running the model version; options: GPU, TPV, VPU or None.

env_param

Optional

List

Environment parameter. See the env_param sample.

framework

Required

String

Calculation framework for running the model version; options: sklearn, tensorflow, pytorch, h2o, spark, etc.

language

Required

String

The language of the development model version; options: python3 or java8.

model_reference

Required

String

Name of the model that the model version belongs to, which is from the model_name_output outputted by the Model operator.

publisher

Required

String

Model creator name.

minio_paths

Optional

List

minio path. See minio_path sample.

Input_data sample


{
  "data": {
    "names": [
      "AGE",
      "RACE",
      "DCAPS",
      "VOL"
    ],
    "ndarray": [
      [
        68,
        2,
        2,
        0
      ]
    ]
  }
}

env_param sample


[
  {
    "name": "string",
    "value": "string",
    "annotations": "string"
  }
]

minio_paths sample


[
  {
    "bucket": "xxx",
    "path": "xxx",
    "destination": "xxx"
  }
]

Output Parameters Description


Parameter

Description

create_model_revision

Whether the model version is built successfully.

model_revision_name

Name of the staged model version.

model_builder_name

Model build name.

Docker Model Version Register Operator


The Docker Model Version Register operator is used to create the model version file and stage the model version by means of container image import. You can select the target files needed to build the model from multiple hierarchical directories of Artifacts source and Git source. When deploying the model, the system will deploy the specified image file as a service.

Input Parameters Description


Name

Required/Optional

Type

Description

input_data

Required

String

Model version parameter input. See the input_data sample.

alias

Optional

String

Alias of the model.

version_rule

Required

String

Model version naming rule, where the time basis is taken by default to name the model version according to the current timestamp.

annotation

Optional

String

Model version description information.

architecture

Required

String

Basic hardware for running model version; options: x86 or arm.

coprocessor

Optional

String

Select the coprocessor for running the model version; options: GPU, TPV, VPU or None.

env_param

Optional

List

Environment parameter. See the env_param sample.

framework

Required

String

Calculation framework for running the model version; options: sklearn, tensorflow, pytorch, h2o, spark, etc.

language

Required

String

Language used for model version development; options: python3 or java8.

model_reference

Required

String

Name of the model that the model version belongs to, which is from the model_name_output outputted by the Model operator.

publisher

Required

String

Model creator name.

minio_paths

Optional

List

minio path. See minio_path sample.

git_setting

Optional

List

Git source setting. See git_setting sample.

docker_dockerfile

Optional

String

Image file name.

git_setting sample


[
  {
    "url": "xxx",
    "user": "xxx",
    "token": "xxx",
    "branch": "xxx",
    "paths": [
      {
        "path": "xxx",
        "destination": "xxx"
      }
    ]
  }
]

Output Parameters Description


Parameter

Description

create_model_revision

Whether the model version is built successfully.

model_revision_name

Name of the staged model version.

model_builder_name

Model build name.

S2i Model Version Register Operator


The S2i Model Version Register operator is used to create the Docker image of model version and stage the model version by using s2i build. You can select the target files needed to build the model from multiple hierarchical directories of Artifacts source and Git source. When deploying the model, the system will deploy the specified image file as a service.

Input Parameters Description


Name

Required/Optional

Type

Description

input_data

Required

String

Model version parameter input. See the input_data sample.

alias

Optional

String

Alias of the model.

version_rule

Required

String

Model version naming rule, where the time basis is taken by default to name the model version according to the current timestamp.

annotation

Optional

String

Model version description information.

architecture

Required

String

Basic hardware for running model version; options: x86 or arm.

coprocessor

Optional

String

Select the coprocessor for running the model version; options: GPU, TPV, VPU or None.

env_param

Optional

List

Environment parameter. See the env_param sample.

framework

Required

String

Calculation framework for running the model version; options: sklearn, tensorflow, pytorch, h2o, spark, etc.

language

Required

String

Language used for model version development; options: python3 or java8.

model_reference

Required

String

Name of the model that the model version belongs to, which is from the model_name_output outputted by the Model operator.

publisher

Required

String

Model creator name.

minio_paths

Optional

List

minio path. See minio_path sample.

git_setting

Optional

List

Git source setting. See git_setting sample.

s2i_model

Optional

String

Name of s2i model file.

Output Parameters Description


Parameter

Description

create_model_revision

Whether the model version is built successfully.

model_revision_name

Name of the staged model version.

model_builder_name

Model build name.

Thirdparty Model Version Register Operator


The Thirdparty Model Version Register operator is used to stage the model version by packaging the model as the Docker image file in the third-party custom system.

Input Parameters Description


Name

Required/Optional

Type

Description

input_data

Required

String

Model version parameter input. See the input_data sample.

alias

Optional

String

Alias of the model.

version_rule

Required

String

Model version naming rule, where the time basis is taken by default to name the model version according to the current timestamp.

annotation

Optional

String

Model version description information.

architecture

Required

String

Basic hardware for running model version; options: x86 or arm.

coprocessor

Optional

String

Select the coprocessor for running the model version; options: GPU, TPV, VPU or None.

env_param

Optional

List

Environment parameter. See the env_param sample.

framework

Required

String

Calculation framework for running the model version; options: sklearn, tensorflow, pytorch, h2o, spark, etc.

language

Required

String

Language used for model version development; options: python3 or java8.

model_reference

Required

String

Name of the model that the model version belongs to, which is from the model_name_output outputted by the Model operator.

publisher

Required

String

Model creator name.

thirdparty_url

Optional

String

Load the address of the model version file.

Output Parameters Description


Parameter

Description

create_model_revision

Whether the model version is built successfully.

model_revision_name

Name of the staged model version.

model_builder_name

Model build name.

Latest Model Version Operator


The Latest Model Version Operator can be used to get the latest model version of a model.

Input Parameters Description


Name

Required/Optional

Type

Description

model_name

Required

model_name

Specify the name of the target model.

serve_as_file

Required

boolean

Choose whether to provide services in the form of model files.

Output Parameters Description


Name

Type

Description

lastest_version

model_version

The latest model version.

Mlflow Model Version Prediction Operator


The Mlflow model version prediction operator is used to perform the prediction based on the machine learning model images.

Input Parameters Description


Name

Required/Optional

Type

Description

model_name

Required

model_name

Model name

model_version_name

Required

model_version

Select the model version.

data

Required

File

Data file.

data_type

Required

data_type

Select the data type, options are csv and json.

Output Parameters Description


Name

Type

Description

predictions

File

Prediction results.

Model Instance Operator


The model instance operator is used to create model deployment instances.

Input Parameters Description


Name

Required/Optional

Type

Description

name

Required

String

Deployment instance name.

resource_pool

Required

String

Name of the resource pool, which comes from the resource name requested through Resource Management.

model_name

Required

String

Model name, which is from the model_name_output outputted by the Model operator.

labels

Optional

List

Tag list.

description

Optional

String

Deployment instance description.

deploy_mode

Required

String

Deployment mode, where only ONLINE (that is, current environment deployment) is supported.

error_on_exist

Required

String

Specify whether to skip without reporting an error when the model name exists. If false is selected, no error will be reported; if `true`is selected and the name of model already exists, an error will be reported directly.

Output Parameters Description


Name

Type

Description

instance_name_output

String

Deployment instance name.

Model Test Operator


The model testing operator is used to test the staged model version.

Input Parameters Description


Name

Required/Optional

Type

Description

input_data

Required

String

Enter the model testing data in JSON format.

model_builder

Required

String

Model builder, which comes from the model_builder_name outputted by any Model Version Register operator.

Output Parameters Description


Name

Type

Description

create_model_test

String

Model test name.

model_test_output

String

Model test return results in JSON format.

Single Model Deployment Operator


The model deployment operator is used to deploy a single model version online.

Input Parameters Description


Name

Required/Optional

Type

Description

model_revision

Required

String

Name of the model version to be deployed, which comes from the model_revision_name outputted by any Model Version Register operator.

instance_name

Required

String

Deployment instance name, which comes from the instance_name_output outputted by the Model Instance operator.

request_cpu

Required

Number

CPU request value required to deploy the model version (the minimum value is 0.01 core).

request_memory

Required

Number

Memory request value required to deploy the model version (the minimum value is 0.1 GB).

limit_cpu

Required

Number

Upper limit of CPU required to deploy the model version (the maximum value is 32 core).

limit_memory

Required

Number

Upper limit of memory required to deploy the model version (the maximum value is 64 GB).

timeout

Required

Number

Timeout setting.

Output Parameters Description


Name

Type

Description

create_model_deployment

String

Name of the deployed model version.

Model Version Info Operator


The Model Version Info operator is used to get the information on model versions in multi-deployment scenarios.

Note

You can only add this operator in the sub-canvas of the Model Deployment operator.

Input Parameters Description


Name

Required/Optional

Type

Description

model_name

Required

model_name

Automatically select the name of the model specified in the model_name parameter of the Model Deployment operator.

model_revision

Required

model_version

Select a model version from the dropdown list.

request_cpu

Required

request_cpu

Specify the value of CPU request.

request_memory

Required

request_memory

Specify the value of memory request.

limit_cpu

Required

limit_cpu

Specify the value of CPU limit.

limit_memory

Required

limit_memory

Specify the value of memory limit.

canary_rate

Optional

number

  • Specify the percentage of traffic to the model version.

  • It only takes effect in the Canary Deployment.

bg_rate

Optional

number

  • Specify the percentage of weighted combination optimization to the model version.

  • It only takes effect in the Blue/Green Deployment.

Output Parameters Description


Name

Type

Description

version_deploy_info

version_deploy_info

The information of the model version to be deployed.

Model Deployment Operator


The Model Deployment operator is used to deploy multiple model versions.

Note

To run this operator, you need to add at least 2 Model Version Info operators to the sub-canvas of the Model Deployment operator.

Input Parameters Description


br />

Name

Required/Optional

Type

Description

model_name

Required

model_name

Select the model to deploy.

instance_name

Required

instance_name

Select the model version instance to deploy.

request_timeout

Required

request_timeout

Enter the timeout threshold for requesting.

deploy_timeout

Optional

deploy_timeout

Enter the timeout threshold for deploying.

using_token

Required

boolean

Choose whether using token.

upgrade_method

Required

upgrade_method

The method to update model versions, supporting Blue/Green, Canary, and Multi-Armed.

combiner_strategy

Required

combiner_strategy

  • The strategy for model combination optimization, supporting Weighted Average and Specify Only.

  • It only takes effect in the Blue/Green Deployment.

specify_version

Required

string

  • Specify the model version for the Specify Only optimization strategy.

  • It only takes effect for the Specify Only optimization strategy.

Output Parameters Description


Name

Type

Description

create_model_deployment

string

The name of the deployment instance.

Service Prediction Operator


Achieve the prediction function based on model deployment service.

Input Parameters Description


Name

Required/Optional

Type

Description

model

Required

String

Model name.

instance

Required

String

Deployment instance name.

namespace

Required

String

Name of the resource pool, which comes from the resource name requested through Resource Management.

data_type

Required

String

Data type. Options: csv or json.
  • The first row of a csv file should be headers.

  • The data in the json file must be complete in the following format:

    {
    "data":
    
    { "names": [ "AGE", "RACE", "DCAPS", "VOL" ], "ndarray": [ [ 0, 0, 0, 0 ] ] }
    
    }
    

data

Required

File

Input data.

Output Parameters Description


Name

Type

Description

predictions

File

Output prediction results.

External Mlflow Prediction Operator


The external Mlflow prediction operator is used to perform the prediction based on the external environment through the model file recorded by Mlflow. The external environment refers to the Jupyter Lab environment in the third-party Lab.

Input Parameters Description


Name

Required/Optional

Type

Description

model_path

Required

Directory

Model file path, which can be derived from the output of the Git Directory operator.

data_type

Required

String

Select the data type, options are csv and json.

data

Required

File

Input data; options: json and csv.

requirements

Optional

List

Content of the requirements.txt file, which is outputted in List format, such as: pandas==1.0.0.

Output Parameters Description


Name

Type

Description

predictions

File

Output prediction results.

Internal Mlflow Prediction Operator


The internal Mlflow prediction operator is used to perform the prediction based on the internal environment through the model file recorded by Mlflow. The internal environment refers to the Jupyter Lab environment in the AI Lab.

Input Parameters Description


Name

Required/Optional

Type

Description

minio_paths

Required

List

Model file path, which is the path of the model file on minio.

data_type

Required

String

Select the data type, options are csv and json.

data

Required

File

Input data; options: json and csv.

requirements

Optional

List

Content of the requirements.txt file, which is outputted in List format, such as: pandas==1.0.0.

Output Parameters Description


Name

Type

Description

predictions

File

Output prediction results.

Model Monitor Operator


The model monitor operator is used to monitor the performance of a specified model in specified period of time. Use the returned metrics result as input of business decisions, such as whether to train the model again or use other algorithms to train new models for replacing the current model version. A model deployment instance can have multiple monitoring indicator, and each model monitor operator can monitor one indicator only.

Input Parameters Description


Name

Required/Optional

Type

Description

model_name

Required

String

Model name.

deployment_instance

Required

String

Model deployment instance name.

start_time

Required

Timestamp

Start time for the model monitoring (selected using the data/time widget, in second unit).

end_time

Required

Timestamp

End time for the model monitoring (selected using the data/time widget, in second unit).

metric_name

Required

String

Monitoring metric name (customized indicator added on the Indicator Monitoring page of AI Hub > Deployment Instance).

Output Parameters Description


Name

Type

Description

model_indicator_data

List

Output model metric monitoring results.

Model Log Operator


The Model Log operator can be used to get up to 5000 logs of a model instance in the last 7 days.

Input Parameters Description


Name

Required/Optional

Type

Description

model_name

Required

String

Specify the Model.

instance_name

Required

model_instance

Specify the model instance.

keyword

Optional

String

Specify the key words to filter model logs.

time_duration

Required

time_duration

Specify the time duration to get model logs.

Output Parameters Description


Name

Type

Description

matched_times

number

The time information of model logs.

log

file

Model log file.