DataWeave can read and write many types of data formats, such as JSON, XML, CSV, XSLS and many others. Before you begin, note that DataWeave version 2 is for Mule 4 apps.
DataWeave supports the following formats as input and output:
| application/avro | avro | Avro Format |
| application/csv | csv | CSV Format |
| application/dw | dw | DataWeave Format (dw) for testing a DataWeave expression |
| application/flatfile | flatfile | Flat File Format, COBOL Copybook Format, Fixed Width Format |
| application/java | java | Java Format, Enum Format for Java |
| application/json | json | JSON Format |
| application/octet-stream | binary | Binary Format |
| application/xlsx | excel | Excel Format |
| application/xml | xml | XML Format, CData Custom Type |
| application/x-ndjson | ndjson | Newline Delimited JSON Format (ndjson) |
| application/x-www-form-urlencoded | urlencoded | URL Encoded Format |
| application/yaml | yaml | YAML Format |
| multipart/form-data | multipart | Multipart Format |
| text/plain | text | Text Plain Format |
| text/x-java-properties | properties | Text Java Properties |
Note: Since DataWeave 2.3, MIME types can be specified with simple IDs such as json or xml. We will work with this syntax throughout this tutorial.
DataWeave Memory Management
DataWeave can read input data as a whole in-memory, in indexed fashion, and for some data formats, part-by-part by streaming the input.
When attempting to read a large file, it is possible to run out of memory or to impact performance negatively. Streaming can improve performance but impacts access to file.
Indexed and In-Memory:
Allow for random access to data because both strategies parse the entire document. When using these strategies, a DataWeave script can access any part of the resulting value at any time.
- Indexed: Uses indexes over the disk. DataWeave can only read the following formats in the indexed mode: CSV, JSON, XML (starting in Mule 4.3.0)
Note that DataWeave does not support reading Excel (XLSX) input in the indexed mode. - In-Memory: Parses the entire document in memory.
DataWeave can read all supported formats using the in-memory mode. - Streaming: Allows for sequential access to the file. This strategy partitions the input document into smaller items and accesses its data sequentially, storing the current item in memory. A DataWeave selector can access the portion of the file that is getting read.
DataWeave supports streaming only for the following formats:
CSV, JSON, Excel (XLSX, starting in Mule 4.2.2), XML (starting in Mule 4.3.0)
Follow Me
If you like my post please follow me to read my latest post on programming and technology.
Leave a Comment