Internal HTTP Client¶
This stage supports HTTP requests through GET, POST, or PUT, and supports caching (generating keys with URL and request parameters) for further queries.
Configuration¶
The configuration tabs for this stage are General, Basic, Input/Output, Request, and QueryConfig.
General¶
Name |
Required? |
Description |
---|---|---|
Name |
Yes |
The name of the stage. |
Description |
No |
The description of the stage. |
Stage Library |
Yes |
The streaming operator library to which the stage belongs. |
Required Fields |
No |
The fields that the data records must contain. If the specified fields are not included, the record will be filtered out. |
Preconditions |
No |
The conditions that must be satisfied by the data records. Records that do not meet the conditions will be filtered out. For example, |
On Record Error |
Yes |
The processing method for error data.
|
Basic¶
Name |
Required? |
Description |
---|---|---|
Enable Cache Fields |
No |
Specify whether to enable caching. Local cache enables fast data processing. When the pipeline is paused, restarted, or retried, the cached data will be lost. |
ExpireRule |
No |
Select the expiring time of the cache. |
Quality Filter |
No |
Filter the data according to the data quality. Only records that meet the quality conditions will be processed by this stage. |
Input/Output¶
Name |
Required? |
Description |
---|---|---|
Input Point |
Yes |
Specify the input point of the records, using the format {modelId}::{pointId}. |
Output Point |
No |
Specify the output point of the records, using the format {modelId}::{pointId}. |
Request¶
Name |
Required? |
Description |
---|---|---|
Request Method |
Yes |
Select the API request method. + Get + Post + Put |
Request URL |
Yes |
Specify the API request URL. |
Param Generate: Position |
Yes |
Specify the position of the parameters and values that are required by the API request. + URL Path + URL Query + Request Body + Header |
Param Generate: Param Key |
Yes |
Specify the parameter name. |
Param Generate: URLParam Type |
Yes |
Select the parameter type. + ByField + ByInput |
Param Generate: Param Value |
Yes |
Specify or select the value of the parameter. |
Access Key |
Yes |
Enter the access key of the application, which is used for API request authentication (the access key can be generated by registering the application on EnOS Management Console). |
Secret Key |
Yes |
Enter the secret key of the application. |
QueryConfig¶
Name |
Required? |
Description |
---|---|---|
Parallel Number |
Yes |
Select the method of parallel requests. Based on the limit of API service, options are System Default and User Defined.
|
Enable Retry |
No |
Specify whether to retry upon request failure, and enter a specific number of retry attempts in the Retry Attempts field. |
Output Results¶
The output results of this stage are included in the lookupTag
field of the attr
struct. The description of the fields are as follows:
Name |
Data Type |
Description |
---|---|---|
hasData |
Boolean |
Indicates whether the API request is successful. |
response |
Object |
If the request is successful, displays the response data. The returnBy field indicates if the response is returned by Redis or HTTP. |
cacheKey |
String |
The request URL composed of |