List of required configuration parameters for the JDBC to S3/ADLS ReadyFlow
When deploying the JDBC to S3/ADLS ReadyFlow, you have to provide the following parameters. Use the information you collected in Prerequisites.
| Parameter name | Description |
|---|---|
| CDP Workload User | Specify the CDP machine user or workload username that you want to use to authenticate to the object stores. Ensure this user has the appropriate access rights to the object store locations in Ranger or IDBroker. |
| CDP Workload User Password | Specify the password of the CDP machine user or workload user you are using to authenticate against the object stores (via IDBroker). |
| CSV Delimiter | If your output data is CSV, specify the delimiter here. |
| Data Output Format | Specify the desired format for your output data. You can use "CSV", "JSON" or "AVRO" with this ReadyFlow. |
| Database Connection URL | Specify the JDBC database connection URL used to connect to your source database. May contain database system name, host, port, database name and some parameters. The exact syntax of a database connection URL is specified by your DBMS. For example, jdbc:mysql://hostname:port/database for MySQL databases. |
| Database Driver | Upload the database driver jar file for your source database. |
| Database Driver Class Name | Specify the source database driver class name. For example, com.mysql.jdbc.Driver for MySQL databases. |
| Database Incremental Fetch Column | Specify the column that is to be used to track state as new data lands in the source table. |
| Database Password | Specify the source database password. |
| Database Table Name | Specify the source database table name. |
| Database User | Specify the source database user. |
| Destination S3 or ADLS Path | Specify the name of the destination S3 or ADLS path you want to write to. Make sure that the path starts with "/". |
| Destination S3 or ADLS Storage Location | Specify the name of the destination S3 bucket or ADLS Container you want to write
to. For S3, enter a value in the form: s3a://[Destination S3 Bucket] For ADLS, enter a value in the form: abfs://[Destination ADLS File System]@[Destination ADLS Storage Account].dfs.core.windows.net |
| Filter Rule | Specify the filter rule expressed in SQL to filter database records. Records matching the filter are written to the destination. The default value forwards all records. |
