Export Flow¶
Export a workflow configuration.
Prerequisite¶
The user must belong to the organization (OU) which the target workflow belongs to.
Request Format¶
GET https://{apigw-address}/batch-processing-service/v2.1/flows
Request Parameters (URI)¶
Name | Location (Path/Query) | Mandatory/Optional | Data Type | Description |
---|---|---|---|---|
flowId | Query | Mandatory | Integer | The workflow ID. |
userId | Query | Mandatory | String | The user ID. How to get userId>> |
orgId | Query | Mandatory | String | The organization ID which the user belongs to. How to get orgId>> |
action | Query | Mandatory | String | Fixed value: export |
Response Parameters¶
Name | Data Type | Description |
---|---|---|
data | Flow Struct | The detailed information of the workflow. For more information, see Flow Struct |
Flow Struct¶
Sample¶
{
"name": "workflow1",
"cycle": "D",
"cron": "0 0 0 * * ? *",
"parameters": "[]",
"alertMode": 3,
"submitter": "yourSubmitter",
"owners": "yourOwners",
"visitors": "yourVisitors;",
"type": 1,
"syncType": 1,
"desc": "",
"startTime": "2019-07-25",
"tasks": [
{
"name": "tass",
"resource": "default",
"type": "COMMAND",
"runMode": "{\"taskMode\":1,\"cpu\":0.5,\"memory\":1,\"maxParallel\":0,\"keyType\":0,\"datasourceId\":0,\"path\":\"\",\"content\":\"\"}",
"syncType": 1,
"cmd": "echo "hello"",
"submitter": "",
"filePackage": "",
"cron": "",
"priorityLevel": 0,
"timeout": 300,
"retryLimit": 3,
"retryInterval": 0,
"successCode": "0",
"asLink": false
}
],
"flowLinks": [],
"taskLinks": [],
"relations": [],
"linkRelations": []
}
Parameters¶
Name | Data Type | Description |
---|---|---|
name | String | The name of the workflow. |
cycle | String | The scheduling cycle.
|
cron | String | If CronTab syntax is used when specifying the time to start the workflow in a scheduling cycle, a 7-character CronTab expression is used. For more information about CronTab, see the Cron Expression Explainer |
parameters | Map | The global parameters that are configured for the workflow to dynamically adapt to the environment changes (in the format of key=value). |
alertMode | Integer | The alert mode.
|
submitter | String | The workflow submission account (the big data account corresponding to the organization). |
owners | String | The username of the owners of the workflow. The semicolon (;) is used to separate each owner, for example, owners=”userNameA;userNameB”. |
visitors | String | The username of the vistors who can access the workflow. The semicolon (;) is used to separate each visitor, for example, visitors=”userNameA;userNameB”. |
type | Integer | The scheduling type of the workflow.
|
syncType | Integer | The sync type of the workflow.
|
desc | String | The workflow description. |
startTime | String | The effective date of the workflow. |
tasks | Array of Task Structs | The collection of task nodes in the workflow, with each element representing a task in the workflow. For more information, see Task Struct |
flowLinks | Array of FlowLink Structs | The collection of workflow references, with each element representing a source workflow that the current workflow depends on. (Using the linkId in the FlowLink Struct, you can find the related link relation in the linkRelations collection. The link relation represents the relationship between a specific task in the current workflow with the source workflow). |
taskLinks | Array of TaskLink Structs | The collection of task references, with each element representing a task in a source workflow that a task in the current workflow depends on. (Using the linkId in the TaskLink Struct, you can find the related link relation in the linkRelations collection. The link relation represents the relationship between a specific task in the current workflow with the source workflow). |
relations | Array of Relation Structs | The collection of relations between tasks, with each element representing the upstream and downstream dependency. For more information, see Relation Struct |
linkRelations | Array of LinkRelation Structs | The collection of link relations. For more information, see LinkRelation Struct |
Task Struct¶
Sample¶
{
"name": "tass",
"resource": "default",
"type": "Shell",
"runMode": "{\"taskMode\":1,\"cpu\":0.5,\"memory\":1,\"maxParallel\":0,\"keyType\":0,\"datasourceId\":0,\"path\":\"\",\"content\":\"\"}",
"syncType": 1,
"cmd": "echo 'hello'",
"submitter": "",
"filePackage": "",
"cron": "",
"priorityLevel": 0,
"timeout": 300,
"retryLimit": 3,
"retryInterval": 0,
"successCode": "0",
"asLink": false
}
Parameters¶
Name | Data Type | Description |
---|---|---|
name | String | The task name. |
resource | String | The task resource. |
type | Integer | The task type (3: SHELL; 7: Data Synchronization; 8: PYTHON; 9: External APP) |
syncType | Integer | The sync type of the task.
|
cmd | String | The command line. |
submitter | String | The workflow submission account (the big data account corresponding to the organization). |
filePackage | String | The file location. |
cron | String | The time for scheduling the task. |
priorityLevel | Integer | The priority level of the task. |
timeout | Integer | The timeout value. |
retryLimit | Integer | The number of retries. |
retryInterval | Integer | The interval for retry. |
successCode | String | The success returned code. |
asLink | Boolean |
|
runMode | String | Task running mode. For more information, see Run Mode Struct |
Run Mode Struct¶
Sample¶
{
"taskMode": 1,
"cpu": 0.5,
"memory": 1,
"maxParallel": 0,
"keyType": 0,
"datasourceId": 0,
"path": "",
"content": ""
}
Parameters¶
Name | Data Type | Description |
---|---|---|
taskMode | Integer | Task running mode (1: Single Task; 2: Multiple Tasks). |
cpu | Float | CPU (unit: core; minimun: 0.1; maximum 2) that is required by running each task (single task or sub-task of multiple tasks). |
memory | Float | Memory (unit: G; minimun: 0.3; maximum 4) that is required by running each task (single task or sub-task of multiple tasks). |
maxParallel | Integer | In multiple tasks mode, the maximum concurrency of sub-tasks. |
keyType | Integer | In multiple tasks mode, the source of the distribution key (1: external files; 2: custom key, specified by the content parameter). |
datasourceId | Integer | When the source of the distribution key is from external files, the data source ID (generated when registering the data source through the Data Source Registration service). |
path | String | When the source of the distribution key is from external files, the path of external data source that stores the distribution key file. |
content | String | When the source of the distribution key is custom key, the content of the distribution key. |
FlowLink Struct¶
Sample¶
{
"linkId": "0",
"sourceFlowName": "jin"
}
Parameters¶
Name | Data Type | Description |
---|---|---|
linkId | String | The link ID. |
sourceFlowName | String | The name of the upstream workflow. |
TaskLink Struct¶
Sample¶
{
"linkId": "0",
"sourceFlowName": "dw",
"sourceTaskName": "tass"
}
Parameters¶
Name | Data Type | Description |
---|---|---|
linkId | String | The link ID. |
sourceFlowName | String | The name of the upstream workflow. |
sourceTaskName | String | The name of the upstream task (the exact upstream task can be found when used with sourceFlowName ). |
Relation Struct¶
Sample¶
{
"sourceTaskName": "tass",
"targetTaskName": "rf",
"rerun": true
}
Parameters¶
Name | Data Type | Description |
---|---|---|
sourceTaskName | String | The name of the upstream task. |
targetTaskName | String | The name of the downstream task. |
rerun | Boolean | Effective only for task-level re-running.
|
LinkRelation Struct¶
Sample¶
{
"linkId": "0",
"targetTaskName": "tass",
"rerun": false
}
Parameters¶
Name | Data Type | Description |
---|---|---|
linkId | String | The link ID. |
targetTaskName | String | The name of the downstream task. |
rerun | Boolean |
|
Error Code¶
Code | Message | Description |
---|---|---|
62102 | Flow validation exception | The format of the request body is invalid. |
62109 | Internal server exception | Internal server exception. |
For other error codes, see Common Error Codes.
Samples¶
Request Sample¶
url: https://{apigw-address}/batch-processing-service/v2.1/flows?action=export&flowId=2924&userId=yourUserId&orgId=yourUserId
method: GET
Return Sample¶
{
"code": 0,
"msg": "OK",
"data": {
"name": "nFlow",
"cycle": "D",
"cron": "0 0 0 * * ? *",
"parameters": "[]",
"alertMode": 3,
"submitter": "yourSubmitter",
"owners": "yourOwners",
"visitors": ";yourVisitors;",
"type": 1,
"syncType": 1,
"desc": "ga",
"startTime": "2019-07-24",
"tasks": [
{
"name": "tass",
"resource": "default",
"type": "COMMAND",
"syncType": 1,
"cmd": "echo 'hello'",
"submitter": "",
"filePackage": "",
"cron": "",
"priorityLevel": 0,
"timeout": 300,
"retryLimit": 3,
"retryInterval": 0,
"successCode": "0",
"asLink": false,
"runMode": "{\"taskMode\":1,\"cpu\":0.5,\"memory\":1,\"maxParallel\":0,\"keyType\":0,\"datasourceId\":0,\"path\":\"\",\"content\":\"\"}"
}
],
"flowLinks": [],
"taskLinks": [],
"relations": [],
"linkRelations": []
}
}