ReadyFlow overview: JDBC to S3/ADLS
You can use the JDBC to S3/ADLS ReadyFlow to move data from a database table to an Amazon S3 bucket or Azure Data Lake Storage location while converting the files to the specified format.
This ReadyFlow consumes data from a source database table. You can filter records by specifying a SQL query in the Filter Rule parameter. The filtered records are then converted to the specified output data format and written to S3 or ADLS. Failed S3 or ADLS write operations are retried automatically to handle transient issues. Define a KPI on the failure_WriteToS3/ADLS connection to monitor failed write operations.
|JDBC to S3/ADLS ReadyFlow details|
|Source Format||Database Table|
|Destination||Amazon S3 or ADLS|
|Destination Format||JSON, CVS, Avro|
Moving data with a JDBC to S3/ADLS flow
This ReadyFlow leverages CDP's centralized access control for cloud storage access. Make sure to either set up an IDBroker mapping or Ranger policies when using fine-grained object store access allowing your workload user access to the target S3 or ADLS locations.