Prerequisites
Learn how to collect the information you need to deploy the Confluent Cloud to Snowflake ReadyFlow, and meet other prerequisites.
For your data ingest source
-
You have created a Confluent Cloud Kafka cluster.
-
You have created at least one Kafka topic.
-
You have created a schema for your data in the Confluent Cloud Schema Registry.
-
You have the Confluent Cloud Kafka broker endpoint.
-
You have the Confluent Cloud Kafka client API Key and Secret.
-
You have the Kafka Consumer Group ID.
-
You have the Confluent Cloud Schema Registry endpoint.
-
You have the Confluent Cloud Schema Registry API Key and Secret.
For DataFlow
-
You have enabled DataFlow for an environment.
For information on how to enable DataFlow for an environment, see Enabling DataFlow for an Environment.
-
You have created a Machine User to use as the CDP Workload User.
- You have given the CDP Workload User the
EnvironmentUser role.
- From the Management Console, go to the environment for which DataFlow is enabled.
- From the Actions drop down, click Manage Access.
- Identify the user you want to use as a Workload User.
- Give that user EnvironmentUser role.
-
You have synchronized your user to the CDP Public Cloud environment that you enabled for DataFlow.
For information on how to synchronize your user to FreeIPA, see Performing User Sync.
- You have granted your CDP user the
DFCatalogAdmin and DFFlowAdmin roles to
enable your user to add the ReadyFlow to the Catalog and deploy the flow definition.
- Give a user permission to add the ReadyFlow to the
Catalog.
- From the Management Console, click User Management.
- Enter the name of the user or group you wish to authorize in the Search field.
- Select the user or group from the list that displays.
- Click .
- From Update Roles, select DFCatalogAdmin and click Update.
- Give your user or group permission to deploy flow definitions.
- From the Management Console, click Environments to display the Environment List page.
- Select the environment to which you want your user or group to deploy flow definitions.
- Click Environment Access page. to display the
- Enter the name of your user or group you wish to authorize in the Search field.
- Select your user or group and click Update Roles.
- Select DFFlowAdmin from the list of roles.
- Click Update Roles.
- Give your user or group access to the Project where the ReadyFlow will be
deployed.
- Go to .
- Select the project where you want to manage access rights and click .
- Start typing the name of the user or group you want to add and select them from the list.
- Select the Resource Roles you want to grant.
- Click Update Roles.
- Click Synchronize Users.
- Give a user permission to add the ReadyFlow to the
Catalog.
For your data ingest target
-
You have created a Snowflake account.
-
Ensure that the Snowflake database and table you are ingesting data to exist. If not, create them.
-
You have set up your destination Snowflake table with the right columns and columns type.
-
Your Snowflake user account has write access to the Snowflake database table.
-
You have the schema of the Snowflake database. The default value is PUBLIC.
- You have created Snowflake Private and Public Keys and assigned the Public Key to your
Snowflake account user.
For more information, see Key Pair Authentication in Snowflake documentation.
- You have created your Snowflake Pipe and configured the file format for CSV input.
For more information, see CREATE FILE FORMAT in Snowflake documentation.