Model Operators¶

The model operators are used to implement various system functions in the MI Pipelines, such as new model creation (analogous to the model registration in the MI Hub), model version staging, model instance creation, single model version online deployment, and prediction based s well as system functions based on the deployed model services or model files. It helps to achieve a closed loop of business from model training to service deployment and prediction based on the trained model.

The MI Pipelines provides the following operators related to registering and deploying machine learning models:

Model (model registration)
Mlflow Model Version Register (model version staging)
Docker Model Version Register (model version staging)
S2i Model Version Register (model version staging)
Thirdparty Model Version Register (model version staging)
Model Test (model testing)
Single Model Deployment (single model version deployment)
Model Instance (creating a model deployment instance)
Service Prediction (prediction based on the model deployment service)
External Mlflow Prediction (prediction with the model file recorded by Mlflow in an external environment)
Internal Mlflow Prediction (prediction with the model file recorded by Mlflow in an internal environment)
Model Monitor (model performance monitoring)
Mlflow Model Version Prediction (prediction based on the machine learning image)

Model Operator¶

The Model operator is used to create a new model (similar to the model registration function in the MI Hub).

Input Parameters Description¶

Name	Required/Optional	Type	Description
category	Required	String	Model category; options: Predictor Combiner Data Transformer
model_name	Required	String	Model name, where lower cases, numbers and dash are supported with the length limited within 32 characters.
alias	Optional	String	Alias of the model.
input_data_type	Required	String	Input data type; options: Text, Tabular or Image.
scope	Required	String	Model application scope; options: Private or Public.
technique	Required	String	Model technology attribution; options: Classification Regression Clustering Other
usecase	Required	String	Model application field; options: Wind Solar Power Storage Smart Factory Smart City Other
publisher	Required	String	Model creator name
input_format	Required	String	Input parameters of the model feature in JSON format. See input_format sample.
output_format	Required	String	Model target output in JSON format. See output_format sample.
interface	Required	String	API type; options: REST or GRPC (that is, the calling method of model service API).
error_on_exist	Optional	String	Specify whether to skip without reporting an error when the model name exists. If false is selected, no error will be reported; if `true`is selected and the name of the model to be created already exists, an error will be reported directly.

input_format sample¶

[
  {
       "name":"X-basic.hour",
       "dtype":"int",
       "ftype":"continuous",
       "range": [
           0,
           23
       ],
       "annotations":"",
       "repeat":null,
       "defaultValue":10
   },
    {
       "name":"X-basic.horizon",
       "dtype":"int",
       "ftype":"continuous",
       "range": [
           0,
           50
       ],
       "annotations":"",
       "repeat":null,
       "defaultValue":8
   }
]

output_format sample¶

[
  {
    "name": "test",
    "dtype": "float",
    "ftype": "continuous",
    "range": [
      1,
      11
    ],
    "annotations": null,
    "repeat": 4,
    "defaultValue": 1
  }
]

Output Parameters Description¶

Name	Type	Description
model_name_output	String	Output the created model name, which is used as the input of Mlflow Model Version Register and other operators.

Mlflow Model Version Register Operator¶

The Mlflow Model Version Register operator is used to create the model version file and stage the model version by means of MLflow import.

Input Parameters Description¶

Name	Required/Optional	Type	Description
input_data	Required	String	Model version parameter input. See the input_data sample.
alias	Optional	String	Alias of the model.
version_rule	Required	String	Model version naming rule, where the time basis is taken by default to name the model version according to the current timestamp.
annotation	Optional	String	Model version description information.
architecture	Required	String	Basic hardware for running model version; options: x86 or arm.
coprocessor	Optional	String	Select the coprocessor for running the model version; options: GPU, TPV, VPU or None.
env_param	Optional	List	Environment parameter. See the env_param sample.
framework	Required	String	Calculation framework for running the model version; options: sklearn, tensorflow, pytorch, h2o, spark, etc.
language	Required	String	The language of the development model version; options: python3 or java8.
model_reference	Required	String	Name of the model that the model version belongs to, which is from the model_name_output outputted by the Model operator.
publisher	Required	String	Model creator name.
minio_paths	Optional	List	minio path. See minio_path sample.

Input_data sample¶

{
  "data": {
    "names": [
      "AGE",
      "RACE",
      "DCAPS",
      "VOL"
    ],
    "ndarray": [
      [
        68,
        2,
        2,
        0
      ]
    ]
  }
}

env_param sample¶

[
  {
    "name": "string",
    "value": "string",
    "annotations": "string"
  }
]

minio_paths sample¶

[
  {
    "bucket": "xxx",
    "path": "xxx",
    "destination": "xxx"
  }
]

Output Parameters Description¶

Parameter	Description
create_model_revision	Whether the model version is built successfully.
model_revision_name	Name of the staged model version.
model_builder_name	Model build name.

Docker Model Version Register Operator¶

The Docker Model Version Register operator is used to create the model version file and stage the model version by means of container image import. You can select the target files needed to build the model from multiple hierarchical directories of Artifacts source and Git source. When deploying the model, the system will deploy the specified image file as a service.

Input Parameters Description¶

Name	Required/Optional	Type	Description
input_data	Required	String	Model version parameter input. See the input_data sample.
alias	Optional	String	Alias of the model.
version_rule	Required	String	Model version naming rule, where the time basis is taken by default to name the model version according to the current timestamp.
annotation	Optional	String	Model version description information.
architecture	Required	String	Basic hardware for running model version; options: x86 or arm.
coprocessor	Optional	String	Select the coprocessor for running the model version; options: GPU, TPV, VPU or None.
env_param	Optional	List	Environment parameter. See the env_param sample.
framework	Required	String	Calculation framework for running the model version; options: sklearn, tensorflow, pytorch, h2o, spark, etc.
language	Required	String	Language used for model version development; options: python3 or java8.
model_reference	Required	String	Name of the model that the model version belongs to, which is from the model_name_output outputted by the Model operator.
publisher	Required	String	Model creator name.
minio_paths	Optional	List	minio path. See minio_path sample.
git_setting	Optional	List	Git source setting. See git_setting sample.
docker_dockerfile	Optional	String	Image file name.

git_setting sample¶

[
  {
    "url": "xxx",
    "user": "xxx",
    "token": "xxx",
    "branch": "xxx",
    "paths": [
      {
        "path": "xxx",
        "destination": "xxx"
      }
    ]
  }
]

Output Parameters Description¶

Parameter	Description
create_model_revision	Whether the model version is built successfully.
model_revision_name	Name of the staged model version.
model_builder_name	Model build name.

S2i Model Version Register Operator¶

The S2i Model Version Register operator is used to create the Docker image of model version and stage the model version by using s2i build. You can select the target files needed to build the model from multiple hierarchical directories of Artifacts source and Git source. When deploying the model, the system will deploy the specified image file as a service.

Input Parameters Description¶

Name	Required/Optional	Type	Description
input_data	Required	String	Model version parameter input. See the input_data sample.
alias	Optional	String	Alias of the model.
version_rule	Required	String	Model version naming rule, where the time basis is taken by default to name the model version according to the current timestamp.
annotation	Optional	String	Model version description information.
architecture	Required	String	Basic hardware for running model version; options: x86 or arm.
coprocessor	Optional	String	Select the coprocessor for running the model version; options: GPU, TPV, VPU or None.
env_param	Optional	List	Environment parameter. See the env_param sample.
framework	Required	String	Calculation framework for running the model version; options: sklearn, tensorflow, pytorch, h2o, spark, etc.
language	Required	String	Language used for model version development; options: python3 or java8.
model_reference	Required	String	Name of the model that the model version belongs to, which is from the model_name_output outputted by the Model operator.
publisher	Required	String	Model creator name.
minio_paths	Optional	List	minio path. See minio_path sample.
git_setting	Optional	List	Git source setting. See git_setting sample.
s2i_model	Optional	String	Name of s2i model file.

Output Parameters Description¶

Parameter	Description
create_model_revision	Whether the model version is built successfully.
model_revision_name	Name of the staged model version.
model_builder_name	Model build name.

Thirdparty Model Version Register Operator¶

The Thirdparty Model Version Register operator is used to stage the model version by packaging the model as the Docker image file in the third-party custom system.

Input Parameters Description¶

Name	Required/Optional	Type	Description
input_data	Required	String	Model version parameter input. See the input_data sample.
alias	Optional	String	Alias of the model.
version_rule	Required	String	Model version naming rule, where the time basis is taken by default to name the model version according to the current timestamp.
annotation	Optional	String	Model version description information.
architecture	Required	String	Basic hardware for running model version; options: x86 or arm.
coprocessor	Optional	String	Select the coprocessor for running the model version; options: GPU, TPV, VPU or None.
env_param	Optional	List	Environment parameter. See the env_param sample.
framework	Required	String	Calculation framework for running the model version; options: sklearn, tensorflow, pytorch, h2o, spark, etc.
language	Required	String	Language used for model version development; options: python3 or java8.
model_reference	Required	String	Name of the model that the model version belongs to, which is from the model_name_output outputted by the Model operator.
publisher	Required	String	Model creator name.
thirdparty_url	Optional	String	Load the address of the model version file.

Output Parameters Description¶

Parameter	Description
create_model_revision	Whether the model version is built successfully.
model_revision_name	Name of the staged model version.
model_builder_name	Model build name.

Model Test Operator¶

The model testing operator is used to test the staged model version.

Input Parameters Description¶

Name	Required/Optional	Type	Description
input_data	Required	String	Enter the model testing data in JSON format.
model_builder	Required	String	Model builder, which comes from the model_builder_name outputted by any Model Version Register operator.

Output Parameters Description¶

Name	Type	Description
create_model_test	String	Model test name.
model_test_output	String	Model test return results in JSON format.

Single Model Deployment Operator¶

The model deployment operator is used to deploy a single model version online.

Input Parameters Description¶

Name	Required/Optional	Type	Description
model_revision	Required	String	Name of the model version to be deployed, which comes from the model_revision_name outputted by any Model Version Register operator.
instance_name	Required	String	Deployment instance name, which comes from the instance_name_output outputted by the Model Instance operator.
request_cpu	Required	Number	CPU request value required to deploy the model version (the minimum value is 0.01 core).
request_memory	Required	Number	Memory request value required to deploy the model version (the minimum value is 0.1 GB).
limit_cpu	Required	Number	Upper limit of CPU required to deploy the model version (the maximum value is 32 core).
limit_memory	Required	Number	Upper limit of memory required to deploy the model version (the maximum value is 64 GB).
timeout	Required	Number	Timeout setting.

Output Parameters Description¶

Name	Type	Description
create_model_deployment	String	Name of the deployed model version.

Model Instance Operator¶

The model instance operator is used to create model deployment instances.

Input Parameters Description¶

Name	Required/Optional	Type	Description
name	Required	String	Deployment instance name.
resource_pool	Required	String	Name of the resource pool, which comes from the resource name requested through Resource Management.
model_name	Required	String	Model name, which is from the model_name_output outputted by the Model operator.
labels	Optional	List	Tag list.
description	Optional	String	Deployment instance description.
deploy_mode	Required	String	Deployment mode, where only ONLINE (that is, current environment deployment) is supported.
error_on_exist	Required	String	Specify whether to skip without reporting an error when the model name exists. If false is selected, no error will be reported; if `true`is selected and the name of model already exists, an error will be reported directly.

Output Parameters Description¶

Name	Type	Description
instance_name_output	String	Deployment instance name.

Service Prediction Operator¶

Achieve the prediction function based on model deployment service.

Input Parameters Description¶

Name	Required/Optional	Type	Description
model	Required	String	Model name.
instance	Required	String	Deployment instance name.
namespace	Required	String	Name of the resource pool, which comes from the resource name requested through Resource Management.
data_type	Required	String	Data type. Options: csv or json. The first row of a csv file should be headers. The data in the json file must be complete in the following format: { "data": { "names": [ "AGE", "RACE", "DCAPS", "VOL" ], "ndarray": [ [ 0, 0, 0, 0 ] ] } }
data	Required	File	Input data.

Output Parameters Description¶

Name	Type	Description
predictions	File	Output prediction results.

External Mlflow Prediction Operator¶

The external Mlflow prediction operator is used to perform the prediction based on the external environment through the model file recorded by Mlflow. The external environment refers to the Jupyter Lab environment in the non-EAP MI Lab.

Input Parameters Description¶

Name	Required/Optional	Type	Description
model_path	Required	Directory	Model file path, which can be derived from the output of the Git Directory operator.
data_type	Required	String	Select the data type, options are `csv` and `json`.
data	Required	File	Input data; options: json and csv.
requirements	Optional	List	Content of the requirements.txt file, which is outputted in List format, such as: pandas==1.0.0.

Output Parameters Description¶

Name	Type	Description
predictions	File	Output prediction results.

Internal Mlflow Prediction Operator¶

The internal Mlflow prediction operator is used to perform the prediction based on the internal environment through the model file recorded by Mlflow. The internal environment refers to the Jupyter Lab environment in the EAP MI Lab.

Input Parameters Description¶

Name	Required/Optional	Type	Description
minio_paths	Required	List	Model file path, which is the path of the model file on minio.
data_type	Required	String	Select the data type, options are `csv` and `json`.
data	Required	File	Input data; options: json and csv.
requirements	Optional	List	Content of the requirements.txt file, which is outputted in List format, such as: pandas==1.0.0.

Output Parameters Description¶

Name	Type	Description
predictions	File	Output prediction results.

Model Monitor Operator¶

The model monitor operator is used to monitor the performance of a specified model in specified period of time. Use the returned metrics result as input of business decisions, such as whether to train the model again or use other algorithms to train new models for replacing the current model version. A model deployment instance can have multiple monitoring indicator, and each model monitor operator can monitor one indicator only.

Input Parameters Description¶

Name	Required/Optional	Type	Description
model_name	Required	String	Model name.
deployment_instance	Required	String	Model deployment instance name.
start_time	Required	Timestamp	Start time for the model monitoring (selected using the data/time widget, in second unit).
end_time	Required	Timestamp	End time for the model monitoring (selected using the data/time widget, in second unit).
metric_name	Required	String	Monitoring metric name (customized indicator added on the Indicator Monitoring page of MI Hub > Deployment Instance).

Output Parameters Description¶

Name	Type	Description
model_indicator_data	List	Output model metric monitoring results.

Mlflow Model Version Prediction Operator¶

The Mlflow model version prediction operator is used to perform the prediction based on the machine learning model images.

Input Parameters Description¶

Name	Required/Optional	Type	Description
model_name	Required	model_name	Model name
model_version_name	Required	model_version	Select the model version.
data	Required	File	Data file.
data_type	Required	data_type	Select the data type, options are `csv` and `json`.

Output Parameters Description¶

Name	Type	Description
predictions	File	Prediction results.