List of required configuration parameters for the S3 to CDW ReadyFlow

When deploying the S3 to CDW ReadyFlow, you have to provide the following parameters. Use the information you collected in Prerequisites.

Table 1. S3 to CDW ReadyFlow configuration parameters
Parameter Name Description
CDP Workload User Specify the CDP machine user or workload user name that you want to use to authenticate to the object stores (via IDBroker) and to the schema registry. Ensure this user has the appropriate access rights to the object store locations and to the schema registry.
CDP Workload User Password Specify the password of the CDP machine user or workload user you are using to authenticate against the object stores (via IDBroker) and the schema registry.
Destination Database Driver Upload the database driver jar file for your destination database.
Destination Database Driver Class Name Specify the destination database driver class name, for example, com.cloudera.impala.jdbc.Driver for Impala databases.
Destination Database Table Name Specify the destination database table name.
Destination S3 Bucket Specify the name of the destination S3 bucket you want to write to. The full path will be constructed out of s3a://#{Destination S3 Bucket}/#{Destination S3 Path}
Destination S3 Path Specify the path within the destination bucket where you want to write to. The full path will be constructed out of s3a://#{Destination S3 Bucket}/#{Destination S3 Path}
Impala JDBC URL Specify the Impala JDBC URL from the CDW Virtual Warehouse. Do not include the UID and PASSWORD connection parameters.

The full JDBC URL will be constructed from:

#{Impala JDBC URL};UID=#{CDP Workload User};PWD=#{CDP Workload User Password}

where PWD value is provided by the dynamic property SENSITIVE.PWD in the Impala connection pool service configuration.

Schema Name Specify the schema name to be looked up in the Schema Registry used to parse the source files.
Schema Registry Hostname Specify the hostname of the Schema Registry you want to connect to. This must be the direct hostname of the Schema Registry itself, not the Knox Endpoint.
Source S3 Bucket Specify the name of the source S3 bucket you want to read from.
Source S3 Path Specify the path within the source bucket where you want to read files from.