ReadyFlow: ListenTCP filter to S3/ADLS

You can use the ListenTCP filter to S3/ADLS ReadyFlow to listen to TCP events on a specified port, filter them, and write them as JSON, CSV or Avro files to S3 or ADLS.

This ReadyFlow listens to a JSON, CSV or Avro data stream on a specified port and parses the data based on a specified Avro-formatted schema. You can filter events by specifying a SQL query in the 'Filter Rule' parameter. The filtered events are then converted to the specified output data format and written to the target S3 or ADLS destination. The flow writes out a file every time its size has either reached 100MB or five minutes have passed. Files can reach a maximum size of 1GB. Failed S3 or ADLS write operations are retried automatically to handle transient issues. Define a KPI on the 'failure_WriteToS3/ADLS' connection to monitor failed write operations.

ReadyFlow details
Source	ListenTCP Processor
Source Format	Avro, JSON, or CSV
Destination	Cloudera managed Amazon S3 or ADLS
Destination Format	Avro, JSON, or CSV