CSV


The CSV node processes a CSV file.


Typically, 1 CU of resource can process a 100M CSV file.

Node Type

Action.

Input and Output Ability

This node has multiple entry points and 1 exit point. The input must be a single CSV file and the output is a JSON array.

Node Properties

../../_images/csv.png


Name

The name for this node.


Method

The method that the node uses to process the CSV file. Currently, the following option is available.

  • Decode: Convert the CSV file into a JSON array.


No. of Header Rows to Ignore

The number of lines from the top of the CSV file to ignore when processing. The first unignored row will be used as the key in the JSON output.


Header

You can choose whether you want to use the default header from the CSV file or create a custom header. The default header is usually the first row of records in the CSV file. If you prefer to have a custom header, you must define your custom header in the box provided.


Delimiter

In order to identify where each field begins and ends in the file, the fields must be separated (delimited) with a character such as a comma (,). You can either choose a delimiter from the drop-down list or define your own by choosing Other. The comma is used as the default delimiter.


Process

The way that this node processes the CSV file. Currently, the following options are available.

  • Line by line: Read, process, and output the data in the CSV file line by line. Supported max file size is 100M. When not required to process the entire file at once, this method is recommended to for better processing performance. To reduce processing time, you can toggle on the Enable Parallel Processing switch.

  • Entire file: Read the file once and output the overall data. Supported max file size is 10M.


Description

The description for this node.

Test CSV File

You can upload your CSV file to see the output sample. Click the Test tab, upload a CSV file, and click the Test button.


The maximum test file size is 2M.


../../_images/csv-test-file.png

Limitations

  • Max node entry points: 100

  • Max file size for line by line: 100M

  • Max file size for entire file: 10M

  • Max test file size: 2M

Samples

Input Sample

A CSV file containing the following information:

assetId, timestamp, temperature, humidity
abc,24214324324,22.4,78
bcd,24214324324,54.4,43

Output Sample

Line by Line

The output for the first line:

{
    "assetId": "abc",
    "timestamp": 24214324324,
    "temperature": 22.4,
    "humidity": 78
}

The output for the second line:

{
    "assetId": "bcd",
    "timestamp": 24214324324,
    "temperature": 54.4,
    "humidity": 43
}

Entire File

[
    {
        "assetId":"abc",
        "timestamp":24214324324,
        "temperature":22.4,
        "humidity":78
    },
    {
        "assetId":"bcd",
        "timestamp":24214324324,
        "temperature":54.4,
        "humidity":43
    }
]