Model Operators¶
The model operators are used to implement various system functions in the AI Pipelines, such as new model creation (analogous to the model registration in the AI Hub), model version staging, model instance creation, single model version online deployment, and prediction based s well as system functions based on the deployed model services or model files. It helps to achieve a closed loop of business from model training to service deployment and prediction based on the trained model.
The AI Pipelines provides the following operators related to registering and deploying machine learning models:
Model (model registration)
Mlflow Model Version Register (model version staging)
Docker Model Version Register (model version staging)
S2i Model Version Register (model version staging)
Thirdparty Model Version Register (model version staging)
Latest Model Version
Mlflow Model Version Prediction (prediction based on the machine learning image)
Model Instance (creating a model deployment instance)
Model Test (model testing)
Single Model Deployment (single model version deployment)
Model Version Info (Getting Model Version Information in multi-deployment scenarios)
Model Deployment (multiple model version deployment)
Service Prediction (prediction based on the model deployment service)
External Mlflow Prediction (prediction with the model file recorded by Mlflow in an external environment)
Internal Mlflow Prediction (prediction with the model file recorded by Mlflow in an internal environment)
Model Monitor (model performance monitoring)
Model Log (Getting model logs by time or key words)
Model Operator¶
The Model operator is used to create a new model (similar to the model registration function in the AI Hub).
Input Parameters Description¶
Name |
Required/Optional |
Type |
Description |
---|---|---|---|
category |
Required |
String |
Model category; options:
|
model_name |
Required |
String |
Model name, where lower cases, numbers and dash are supported with the length limited within 32 characters. |
alias |
Optional |
String |
Alias of the model. |
input_data_type |
Required |
String |
Input data type; options: Text, Tabular or Image. |
scope |
Required |
String |
Model application scope; options: Private or Public. |
technique |
Required |
String |
Model technology attribution; options:
|
usecase |
Required |
String |
Model application field; options:
|
publisher |
Required |
String |
Model creator name |
input_format |
Required |
String |
Input parameters of the model feature in JSON format. See input_format sample. |
output_format |
Required |
String |
Model target output in JSON format. See output_format sample. |
interface |
Required |
String |
API type; options: REST or GRPC (that is, the calling method of model service API). |
error_on_exist |
Optional |
Boolean |
Specify whether to skip without reporting an error when the model name exists. If false is selected, no error will be reported; if `true`is selected and the name of the model to be created already exists, an error will be reported directly. |
input_format sample¶
[
{
"name":"X-basic.hour",
"dtype":"int",
"ftype":"continuous",
"range": [
0,
23
],
"annotations":"",
"repeat":null,
"defaultValue":10
},
{
"name":"X-basic.horizon",
"dtype":"int",
"ftype":"continuous",
"range": [
0,
50
],
"annotations":"",
"repeat":null,
"defaultValue":8
}
]
output_format sample¶
[
{
"name": "test",
"dtype": "float",
"ftype": "continuous",
"range": [
1,
11
],
"annotations": null,
"repeat": 4,
"defaultValue": 1
}
]
Output Parameters Description¶
Name |
Type |
Description |
---|---|---|
model_name_output |
String |
Output the created model name, which is used as the input of Mlflow Model Version Register and other operators. |
Mlflow Model Version Register Operator¶
The Mlflow Model Version Register operator is used to create the model version file and stage the model version by means of MLflow import.
Input Parameters Description¶
Name |
Required/Optional |
Type |
Description |
---|---|---|---|
input_data |
Required |
String |
Model version parameter input. See the input_data sample. |
alias |
Optional |
String |
Alias of the model. |
version_rule |
Required |
String |
Model version naming rule, where the time basis is taken by default to name the model version according to the current timestamp. |
annotation |
Optional |
String |
Model version description information. |
architecture |
Required |
String |
Basic hardware for running model version; options: x86 or arm. |
coprocessor |
Optional |
String |
Select the coprocessor for running the model version; options: GPU, TPV, VPU or None. |
env_param |
Optional |
List |
Environment parameter. See the env_param sample. |
framework |
Required |
String |
Calculation framework for running the model version; options: sklearn, tensorflow, pytorch, h2o, spark, etc. |
language |
Required |
String |
The language of the development model version; options: python3 or java8. |
model_reference |
Required |
String |
Name of the model that the model version belongs to, which is from the model_name_output outputted by the Model operator. |
publisher |
Required |
String |
Model creator name. |
minio_paths |
Optional |
List |
minio path. See minio_path sample. |
Input_data sample¶
{
"data": {
"names": [
"AGE",
"RACE",
"DCAPS",
"VOL"
],
"ndarray": [
[
68,
2,
2,
0
]
]
}
}
env_param sample¶
[
{
"name": "string",
"value": "string",
"annotations": "string"
}
]
minio_paths sample¶
[
{
"bucket": "xxx",
"path": "xxx",
"destination": "xxx"
}
]
Output Parameters Description¶
Parameter |
Description |
---|---|
create_model_revision |
Whether the model version is built successfully. |
model_revision_name |
Name of the staged model version. |
model_builder_name |
Model build name. |
Docker Model Version Register Operator¶
The Docker Model Version Register operator is used to create the model version file and stage the model version by means of container image import. You can select the target files needed to build the model from multiple hierarchical directories of Artifacts source and Git source. When deploying the model, the system will deploy the specified image file as a service.
Input Parameters Description¶
Name |
Required/Optional |
Type |
Description |
---|---|---|---|
input_data |
Required |
String |
Model version parameter input. See the input_data sample. |
alias |
Optional |
String |
Alias of the model. |
version_rule |
Required |
String |
Model version naming rule, where the time basis is taken by default to name the model version according to the current timestamp. |
annotation |
Optional |
String |
Model version description information. |
architecture |
Required |
String |
Basic hardware for running model version; options: x86 or arm. |
coprocessor |
Optional |
String |
Select the coprocessor for running the model version; options: GPU, TPV, VPU or None. |
env_param |
Optional |
List |
Environment parameter. See the env_param sample. |
framework |
Required |
String |
Calculation framework for running the model version; options: sklearn, tensorflow, pytorch, h2o, spark, etc. |
language |
Required |
String |
Language used for model version development; options: python3 or java8. |
model_reference |
Required |
String |
Name of the model that the model version belongs to, which is from the model_name_output outputted by the Model operator. |
publisher |
Required |
String |
Model creator name. |
minio_paths |
Optional |
List |
minio path. See minio_path sample. |
git_setting |
Optional |
List |
Git source setting. See git_setting sample. |
docker_dockerfile |
Optional |
String |
Image file name. |
git_setting sample¶
[
{
"url": "xxx",
"user": "xxx",
"token": "xxx",
"branch": "xxx",
"paths": [
{
"path": "xxx",
"destination": "xxx"
}
]
}
]
Output Parameters Description¶
Parameter |
Description |
---|---|
create_model_revision |
Whether the model version is built successfully. |
model_revision_name |
Name of the staged model version. |
model_builder_name |
Model build name. |
S2i Model Version Register Operator¶
The S2i Model Version Register operator is used to create the Docker image of model version and stage the model version by using s2i build. You can select the target files needed to build the model from multiple hierarchical directories of Artifacts source and Git source. When deploying the model, the system will deploy the specified image file as a service.
Input Parameters Description¶
Name |
Required/Optional |
Type |
Description |
---|---|---|---|
input_data |
Required |
String |
Model version parameter input. See the input_data sample. |
alias |
Optional |
String |
Alias of the model. |
version_rule |
Required |
String |
Model version naming rule, where the time basis is taken by default to name the model version according to the current timestamp. |
annotation |
Optional |
String |
Model version description information. |
architecture |
Required |
String |
Basic hardware for running model version; options: x86 or arm. |
coprocessor |
Optional |
String |
Select the coprocessor for running the model version; options: GPU, TPV, VPU or None. |
env_param |
Optional |
List |
Environment parameter. See the env_param sample. |
framework |
Required |
String |
Calculation framework for running the model version; options: sklearn, tensorflow, pytorch, h2o, spark, etc. |
language |
Required |
String |
Language used for model version development; options: python3 or java8. |
model_reference |
Required |
String |
Name of the model that the model version belongs to, which is from the model_name_output outputted by the Model operator. |
publisher |
Required |
String |
Model creator name. |
minio_paths |
Optional |
List |
minio path. See minio_path sample. |
git_setting |
Optional |
List |
Git source setting. See git_setting sample. |
s2i_model |
Optional |
String |
Name of s2i model file. |
Output Parameters Description¶
Parameter |
Description |
---|---|
create_model_revision |
Whether the model version is built successfully. |
model_revision_name |
Name of the staged model version. |
model_builder_name |
Model build name. |
Thirdparty Model Version Register Operator¶
The Thirdparty Model Version Register operator is used to stage the model version by packaging the model as the Docker image file in the third-party custom system.
Input Parameters Description¶
Name |
Required/Optional |
Type |
Description |
---|---|---|---|
input_data |
Required |
String |
Model version parameter input. See the input_data sample. |
alias |
Optional |
String |
Alias of the model. |
version_rule |
Required |
String |
Model version naming rule, where the time basis is taken by default to name the model version according to the current timestamp. |
annotation |
Optional |
String |
Model version description information. |
architecture |
Required |
String |
Basic hardware for running model version; options: x86 or arm. |
coprocessor |
Optional |
String |
Select the coprocessor for running the model version; options: GPU, TPV, VPU or None. |
env_param |
Optional |
List |
Environment parameter. See the env_param sample. |
framework |
Required |
String |
Calculation framework for running the model version; options: sklearn, tensorflow, pytorch, h2o, spark, etc. |
language |
Required |
String |
Language used for model version development; options: python3 or java8. |
model_reference |
Required |
String |
Name of the model that the model version belongs to, which is from the model_name_output outputted by the Model operator. |
publisher |
Required |
String |
Model creator name. |
thirdparty_url |
Optional |
String |
Load the address of the model version file. |
Output Parameters Description¶
Parameter |
Description |
---|---|
create_model_revision |
Whether the model version is built successfully. |
model_revision_name |
Name of the staged model version. |
model_builder_name |
Model build name. |
Latest Model Version Operator¶
The Latest Model Version Operator can be used to get the latest model version of a model.
Input Parameters Description¶
Name |
Required/Optional |
Type |
Description |
---|---|---|---|
model_name |
Required |
model_name |
Specify the name of the target model. |
serve_as_file |
Required |
boolean |
Choose whether to provide services in the form of model files. |
Output Parameters Description¶
Name |
Type |
Description |
---|---|---|
lastest_version |
model_version |
The latest model version. |
Mlflow Model Version Prediction Operator¶
The Mlflow model version prediction operator is used to perform the prediction based on the machine learning model images.
Input Parameters Description¶
Name |
Required/Optional |
Type |
Description |
---|---|---|---|
model_name |
Required |
model_name |
Model name |
model_version_name |
Required |
model_version |
Select the model version. |
data |
Required |
File |
Data file. |
data_type |
Required |
data_type |
Select the data type, options are |
Output Parameters Description¶
Name |
Type |
Description |
---|---|---|
predictions |
File |
Prediction results. |
Model Instance Operator¶
The model instance operator is used to create model deployment instances.
Input Parameters Description¶
Name |
Required/Optional |
Type |
Description |
---|---|---|---|
name |
Required |
String |
Deployment instance name. |
resource_pool |
Required |
String |
Name of the resource pool, which comes from the resource name requested through Resource Management. |
model_name |
Required |
String |
Model name, which is from the model_name_output outputted by the Model operator. |
labels |
Optional |
List |
Tag list. |
description |
Optional |
String |
Deployment instance description. |
deploy_mode |
Required |
String |
Deployment mode, where only ONLINE (that is, current environment deployment) is supported. |
error_on_exist |
Required |
String |
Specify whether to skip without reporting an error when the model name exists. If false is selected, no error will be reported; if `true`is selected and the name of model already exists, an error will be reported directly. |
Output Parameters Description¶
Name |
Type |
Description |
---|---|---|
instance_name_output |
String |
Deployment instance name. |
Model Test Operator¶
The model testing operator is used to test the staged model version.
Input Parameters Description¶
Name |
Required/Optional |
Type |
Description |
---|---|---|---|
input_data |
Required |
String |
Enter the model testing data in JSON format. |
model_builder |
Required |
String |
Model builder, which comes from the model_builder_name outputted by any Model Version Register operator. |
Output Parameters Description¶
Name |
Type |
Description |
---|---|---|
create_model_test |
String |
Model test name. |
model_test_output |
String |
Model test return results in JSON format. |
Single Model Deployment Operator¶
The model deployment operator is used to deploy a single model version online.
Input Parameters Description¶
Name |
Required/Optional |
Type |
Description |
---|---|---|---|
model_revision |
Required |
String |
Name of the model version to be deployed, which comes from the model_revision_name outputted by any Model Version Register operator. |
instance_name |
Required |
String |
Deployment instance name, which comes from the instance_name_output outputted by the Model Instance operator. |
request_cpu |
Required |
Number |
CPU request value required to deploy the model version (the minimum value is 0.01 core). |
request_memory |
Required |
Number |
Memory request value required to deploy the model version (the minimum value is 0.1 GB). |
limit_cpu |
Required |
Number |
Upper limit of CPU required to deploy the model version (the maximum value is 32 core). |
limit_memory |
Required |
Number |
Upper limit of memory required to deploy the model version (the maximum value is 64 GB). |
timeout |
Required |
Number |
Timeout setting. |
Output Parameters Description¶
Name |
Type |
Description |
---|---|---|
create_model_deployment |
String |
Name of the deployed model version. |
Model Version Info Operator¶
The Model Version Info operator is used to get the information on model versions in multi-deployment scenarios.
Note
You can only add this operator in the sub-canvas of the Model Deployment operator.
Input Parameters Description¶
Name |
Required/Optional |
Type |
Description |
---|---|---|---|
model_name |
Required |
model_name |
Automatically select the name of the model specified in the model_name parameter of the Model Deployment operator. |
model_revision |
Required |
model_version |
Select a model version from the dropdown list. |
request_cpu |
Required |
request_cpu |
Specify the value of CPU request. |
request_memory |
Required |
request_memory |
Specify the value of memory request. |
limit_cpu |
Required |
limit_cpu |
Specify the value of CPU limit. |
limit_memory |
Required |
limit_memory |
Specify the value of memory limit. |
canary_rate |
Optional |
number |
|
bg_rate |
Optional |
number |
|
Output Parameters Description¶
Name |
Type |
Description |
---|---|---|
version_deploy_info |
version_deploy_info |
The information of the model version to be deployed. |
Model Deployment Operator¶
The Model Deployment operator is used to deploy multiple model versions.
Note
To run this operator, you need to add at least 2 Model Version Info operators to the sub-canvas of the Model Deployment operator.
Input Parameters Description¶
br />
Name |
Required/Optional |
Type |
Description |
---|---|---|---|
model_name |
Required |
model_name |
Select the model to deploy. |
instance_name |
Required |
instance_name |
Select the model version instance to deploy. |
request_timeout |
Required |
request_timeout |
Enter the timeout threshold for requesting. |
deploy_timeout |
Optional |
deploy_timeout |
Enter the timeout threshold for deploying. |
using_token |
Required |
boolean |
Choose whether using token. |
upgrade_method |
Required |
upgrade_method |
The method to update model versions, supporting Blue/Green, Canary, and Multi-Armed. |
combiner_strategy |
Required |
combiner_strategy |
|
specify_version |
Required |
string |
|
Output Parameters Description¶
Name |
Type |
Description |
---|---|---|
create_model_deployment |
string |
The name of the deployment instance. |
Service Prediction Operator¶
Achieve the prediction function based on model deployment service.
Input Parameters Description¶
Name |
Required/Optional |
Type |
Description |
---|---|---|---|
model |
Required |
String |
Model name. |
instance |
Required |
String |
Deployment instance name. |
namespace |
Required |
String |
Name of the resource pool, which comes from the resource name requested through Resource Management. |
data_type |
Required |
String |
|
data |
Required |
File |
Input data. |
Output Parameters Description¶
Name |
Type |
Description |
---|---|---|
predictions |
File |
Output prediction results. |
External Mlflow Prediction Operator¶
The external Mlflow prediction operator is used to perform the prediction based on the external environment through the model file recorded by Mlflow. The external environment refers to the Jupyter Lab environment in the third-party Lab.
Input Parameters Description¶
Name |
Required/Optional |
Type |
Description |
---|---|---|---|
model_path |
Required |
Directory |
Model file path, which can be derived from the output of the Git Directory operator. |
data_type |
Required |
String |
Select the data type, options are |
data |
Required |
File |
Input data; options: json and csv. |
requirements |
Optional |
List |
Content of the requirements.txt file, which is outputted in List format, such as: pandas==1.0.0. |
Output Parameters Description¶
Name |
Type |
Description |
---|---|---|
predictions |
File |
Output prediction results. |
Internal Mlflow Prediction Operator¶
The internal Mlflow prediction operator is used to perform the prediction based on the internal environment through the model file recorded by Mlflow. The internal environment refers to the Jupyter Lab environment in the AI Lab.
Input Parameters Description¶
Name |
Required/Optional |
Type |
Description |
---|---|---|---|
minio_paths |
Required |
List |
Model file path, which is the path of the model file on minio. |
data_type |
Required |
String |
Select the data type, options are |
data |
Required |
File |
Input data; options: json and csv. |
requirements |
Optional |
List |
Content of the requirements.txt file, which is outputted in List format, such as: pandas==1.0.0. |
Output Parameters Description¶
Name |
Type |
Description |
---|---|---|
predictions |
File |
Output prediction results. |
Model Monitor Operator¶
The model monitor operator is used to monitor the performance of a specified model in specified period of time. Use the returned metrics result as input of business decisions, such as whether to train the model again or use other algorithms to train new models for replacing the current model version. A model deployment instance can have multiple monitoring indicator, and each model monitor operator can monitor one indicator only.
Input Parameters Description¶
Name |
Required/Optional |
Type |
Description |
---|---|---|---|
model_name |
Required |
String |
Model name. |
deployment_instance |
Required |
String |
Model deployment instance name. |
start_time |
Required |
Timestamp |
Start time for the model monitoring (selected using the data/time widget, in second unit). |
end_time |
Required |
Timestamp |
End time for the model monitoring (selected using the data/time widget, in second unit). |
metric_name |
Required |
String |
Monitoring metric name (customized indicator added on the Indicator Monitoring page of AI Hub > Deployment Instance). |
Output Parameters Description¶
Name |
Type |
Description |
---|---|---|
model_indicator_data |
List |
Output model metric monitoring results. |
Model Log Operator¶
The Model Log operator can be used to get up to 5000 logs of a model instance in the last 7 days.
Input Parameters Description¶
Name |
Required/Optional |
Type |
Description |
---|---|---|---|
model_name |
Required |
String |
Specify the Model. |
instance_name |
Required |
model_instance |
Specify the model instance. |
keyword |
Optional |
String |
Specify the key words to filter model logs. |
time_duration |
Required |
time_duration |
Specify the time duration to get model logs. |
Output Parameters Description¶
Name |
Type |
Description |
---|---|---|
matched_times |
number |
The time information of model logs. |
log |
file |
Model log file. |