Compresses or decompresses the contents of FlowFiles using a user-specified compression algorithm and updates the mime.type attribute as appropriate. A common idiom is to precede CompressContent with IdentifyMimeType and configure Mode='decompress' AND Compression Format='use mime.type attribute'. When used in this manner, the MIME type is automatically detected and the data is decompressed, if necessary. If decompression is unnecessary, the data is passed through to the 'success' relationship. This processor operates in a very memory efficient way so very large objects well beyond the heap size are generally fine to process.
content, compress, decompress, gzip, bzip2, lzma, xz-lzma2, snappy, snappy-hadoop, snappy framed, lz4-framed, deflate, zstd, brotli
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values.
| Display Name | API Name | Default Value | Allowable Values | Description |
|---|---|---|---|---|
| Mode | Mode | compress |
| Indicates whether the processor should compress content or decompress content. Must be either 'compress' or 'decompress' |
| Compression Format | Compression Format | use mime.type attribute |
| The compression format to use. Valid values are: GZIP, Deflate, ZSTD, BZIP2, XZ-LZMA2, LZMA, Brotli, Snappy, Snappy Hadoop, Snappy Framed, and LZ4-Framed |
| Compression Level | Compression Level | 1 |
| The compression level to use; this is valid only when using gzip, deflate or xz-lzma2 compression. A lower value results in faster processing but less compression; a value of 0 indicates no (that is, simple archiving) for gzip or minimal for xz-lzma2 compression. Higher levels can mean much larger memory usage such as the case with levels 7-9 for xz-lzma/2 so be careful relative to heap size. This Property is only considered if all of the following conditions are met:
|
| Update Filename | Update Filename | false |
| If true, will remove the filename extension when decompressing data (only if the extension indicates the appropriate compression format) and add the appropriate extension when compressing data |
| Name | Description |
|---|---|
| success | FlowFiles will be transferred to the success relationship after successfully being compressed or decompressed |
| failure | FlowFiles will be transferred to the failure relationship if they fail to compress/decompress |
| Name | Description |
|---|---|
| mime.type | If the Compression Format is set to use mime.type attribute, this attribute is used to determine the compression type. Otherwise, this attribute is ignored. |
| Name | Description |
|---|---|
| mime.type | If the Mode property is set to compress, the appropriate MIME Type is set. If the Mode property is set to decompress and the file is successfully decompressed, this attribute is removed, as the MIME Type is no longer known. |
Compress the contents of a FlowFile
"Mode" = "compress"
"Compression Format" should be set to whichever compression algorithm should be used.
Decompress the contents of a FlowFile
"Mode" = "decompress"
"Compression Format" should be set to whichever compression algorithm was used to compress the data previously.
Check whether or not a FlowFile is compressed and if so, decompress it.
If IdentifyMimeType determines that the content is not compressed, CompressContent will pass the FlowFile along to the 'success' relationship without attempting to decompress it.
Default property values are sufficient.
Connect the 'success' relationship to CompressContent.
"Mode" = "decompress"
"Compression Format" = "use mime.type attribute"
| Resource | Description |
|---|---|
| CPU | An instance of this component can cause high usage of this system resource. Multiple instances or high concurrency settings may result a degradation of performance. |
| MEMORY | An instance of this component can cause high usage of this system resource. Multiple instances or high concurrency settings may result a degradation of performance. |