Export Flow¶
Export a workflow configuration.
Prerequisite¶
The user must belong to the organization (OU) which the target workflow belongs to.
Request Format¶
GET https://{apigw-address}/batch-processing-service/v2.1/flows
Request Parameters (URI)¶
Name |
Location (Path/Query) |
Mandatory/Optional |
Data Type |
Description |
---|---|---|---|---|
flowId |
Query |
Mandatory |
Integer |
The workflow ID. |
userId |
Query |
Mandatory |
String |
The user ID. How to get userId>> |
orgId |
Query |
Mandatory |
String |
The organization ID which the user belongs to. How to get orgId>> |
action |
Query |
Mandatory |
String |
Fixed value: export |
Response Parameters¶
Name |
Data Type |
Description |
---|---|---|
data |
Flow Struct |
The detailed information of the workflow. For more information, see Flow Struct |
Flow Struct¶
Sample¶
{
"name": "workflow1",
"cycle": "D",
"cron": "0 0 0 * * ? *",
"parameters": "[]",
"alertMode": 3,
"submitter": "yourSubmitter",
"owners": "yourOwners",
"visitors": "yourVisitors;",
"type": 1,
"syncType": 1,
"desc": "",
"startTime": "2019-07-25",
"tasks": [
{
"name": "tass",
"resource": "default",
"type": "COMMAND",
"runMode": "{\"taskMode\":1,\"cpu\":0.5,\"memory\":1,\"maxParallel\":0,\"keyType\":0,\"datasourceId\":0,\"path\":\"\",\"content\":\"\"}",
"syncType": 1,
"cmd": "echo "hello"",
"submitter": "",
"filePackage": "",
"cron": "",
"priorityLevel": 0,
"timeout": 300,
"retryLimit": 3,
"retryInterval": 0,
"successCode": "0",
"asLink": false
}
],
"flowLinks": [],
"taskLinks": [],
"relations": [],
"linkRelations": []
}
Parameters¶
Name |
Data Type |
Description |
---|---|---|
name |
String |
The name of the workflow. |
cycle |
String |
The scheduling cycle.
|
cron |
String |
If CronTab syntax is used when specifying the time to start the workflow in a scheduling cycle, a 7-character CronTab expression is used. For more information about CronTab, see the Cron Expression Explainer |
parameters |
Map |
The global parameters that are configured for the workflow to dynamically adapt to the environment changes (in the format of key=value). |
alertMode |
Integer |
The alert mode.
|
submitter |
String |
The workflow submission account (the big data account corresponding to the organization). |
owners |
String |
The username of the owners of the workflow. The semicolon (;) is used to separate each owner, for example, owners=”userNameA;userNameB”. |
visitors |
String |
The username of the vistors who can access the workflow. The semicolon (;) is used to separate each visitor, for example, visitors=”userNameA;userNameB”. |
type |
Integer |
The scheduling type of the workflow.
|
syncType |
Integer |
The sync type of the workflow.
|
desc |
String |
The workflow description. |
startTime |
String |
The effective date of the workflow. |
tasks |
Array of Task Structs |
The collection of task nodes in the workflow, with each element representing a task in the workflow. For more information, see Task Struct |
flowLinks |
Array of FlowLink Structs |
The collection of workflow references, with each element representing a source workflow that the current workflow depends on. (Using the |
taskLinks |
Array of TaskLink Structs |
The collection of task references, with each element representing a task in a source workflow that a task in the current workflow depends on. (Using the |
relations |
Array of Relation Structs |
The collection of relations between tasks, with each element representing the upstream and downstream dependency. For more information, see Relation Struct |
linkRelations |
Array of LinkRelation Structs |
The collection of link relations. For more information, see LinkRelation Struct |
Task Struct¶
Sample¶
{
"name": "tass",
"resource": "default",
"type": "Shell",
"runMode": "{\"taskMode\":1,\"cpu\":0.5,\"memory\":1,\"maxParallel\":0,\"keyType\":0,\"datasourceId\":0,\"path\":\"\",\"content\":\"\"}",
"syncType": 1,
"cmd": "echo 'hello'",
"submitter": "",
"filePackage": "",
"cron": "",
"priorityLevel": 0,
"timeout": 300,
"retryLimit": 3,
"retryInterval": 0,
"successCode": "0",
"asLink": false
}
Parameters¶
Name |
Data Type |
Description |
---|---|---|
name |
String |
The task name. |
resource |
String |
The task resource. |
type |
Integer |
The task type (3: SHELL; 7: Data Synchronization; 8: PYTHON; 9: External APP) |
syncType |
Integer |
The sync type of the task.
|
cmd |
String |
The command line. |
submitter |
String |
The workflow submission account (the big data account corresponding to the organization). |
filePackage |
String |
The file location. |
cron |
String |
The time for scheduling the task. |
priorityLevel |
Integer |
The priority level of the task. |
timeout |
Integer |
The timeout value. |
retryLimit |
Integer |
The number of retries. |
retryInterval |
Integer |
The interval for retry. |
successCode |
String |
The success returned code. |
asLink |
Boolean |
|
runMode |
String |
Task running mode. For more information, see Run Mode Struct |
Run Mode Struct¶
Sample¶
{
"taskMode": 1,
"cpu": 0.5,
"memory": 1,
"maxParallel": 0,
"keyType": 0,
"datasourceId": 0,
"path": "",
"content": ""
}
Parameters¶
Name |
Data Type |
Description |
---|---|---|
taskMode |
Integer |
Task running mode (1: Single Task; 2: Multiple Tasks). |
cpu |
Float |
CPU (unit: core; minimun: 0.1; maximum 2) that is required by running each task (single task or sub-task of multiple tasks). |
memory |
Float |
Memory (unit: G; minimun: 0.3; maximum 4) that is required by running each task (single task or sub-task of multiple tasks). |
maxParallel |
Integer |
In multiple tasks mode, the maximum concurrency of sub-tasks. |
keyType |
Integer |
In multiple tasks mode, the source of the distribution key (1: external files; 2: custom key, specified by the |
datasourceId |
Integer |
When the source of the distribution key is from external files, the data source ID (generated when registering the data source through the Data Source Registration service). |
path |
String |
When the source of the distribution key is from external files, the path of external data source that stores the distribution key file. |
content |
String |
When the source of the distribution key is custom key, the content of the distribution key. |
FlowLink Struct¶
Sample¶
{
"linkId": "0",
"sourceFlowName": "jin"
}
Parameters¶
Name |
Data Type |
Description |
---|---|---|
linkId |
String |
The link ID. |
sourceFlowName |
String |
The name of the upstream workflow. |
TaskLink Struct¶
Sample¶
{
"linkId": "0",
"sourceFlowName": "dw",
"sourceTaskName": "tass"
}
Parameters¶
Name |
Data Type |
Description |
---|---|---|
linkId |
String |
The link ID. |
sourceFlowName |
String |
The name of the upstream workflow. |
sourceTaskName |
String |
The name of the upstream task (the exact upstream task can be found when used with |
Relation Struct¶
Sample¶
{
"sourceTaskName": "tass",
"targetTaskName": "rf",
"rerun": true
}
Parameters¶
Name |
Data Type |
Description |
---|---|---|
sourceTaskName |
String |
The name of the upstream task. |
targetTaskName |
String |
The name of the downstream task. |
rerun |
Boolean |
Effective only for task-level re-running.
|
LinkRelation Struct¶
Sample¶
{
"linkId": "0",
"targetTaskName": "tass",
"rerun": false
}
Parameters¶
Name |
Data Type |
Description |
---|---|---|
linkId |
String |
The link ID. |
targetTaskName |
String |
The name of the downstream task. |
rerun |
Boolean |
|
Error Code¶
Code |
Message |
Description |
---|---|---|
62102 |
Flow validation exception |
The format of the request body is invalid. |
62109 |
Internal server exception |
Internal server exception. |
For other error codes, see Common Error Codes.
Samples¶
Request Sample¶
url: https://{apigw-address}/batch-processing-service/v2.1/flows?action=export&flowId=2924&userId=yourUserId&orgId=yourUserId
method: GET
Return Sample¶
{
"code": 0,
"msg": "OK",
"data": {
"name": "nFlow",
"cycle": "D",
"cron": "0 0 0 * * ? *",
"parameters": "[]",
"alertMode": 3,
"submitter": "yourSubmitter",
"owners": "yourOwners",
"visitors": ";yourVisitors;",
"type": 1,
"syncType": 1,
"desc": "ga",
"startTime": "2019-07-24",
"tasks": [
{
"name": "tass",
"resource": "default",
"type": "COMMAND",
"syncType": 1,
"cmd": "echo 'hello'",
"submitter": "",
"filePackage": "",
"cron": "",
"priorityLevel": 0,
"timeout": 300,
"retryLimit": 3,
"retryInterval": 0,
"successCode": "0",
"asLink": false,
"runMode": "{\"taskMode\":1,\"cpu\":0.5,\"memory\":1,\"maxParallel\":0,\"keyType\":0,\"datasourceId\":0,\"path\":\"\",\"content\":\"\"}"
}
],
"flowLinks": [],
"taskLinks": [],
"relations": [],
"linkRelations": []
}
}
SDK Samples¶
You can access the SDK samples for batch data processing service on GitHub: