Cloudera Docs

Setting up connectors that use JDBC drivers

The Debezium and JDBC Kafka Connect connectors shipped with CSP Community Edition require a JDBC driver to function. However, by default only the PostgreSQL JDBC driver is available for use. If you want to deploy a connector that requires a different JDBC driver to function, you must copy the driver to the Kafka Connect container before you deploy the connector.

The following Kafka Connect connectors require a JDBC driver that does not come shipped with CSP Community Edition:

  • Debezium Db2 Source
  • Debezium MySQL Source
  • Debezium Oracle Source
  • Debezium SQL Server Source
  • JDBC Source
  • JDBC Sink

Before you can deploy any of these connectors you must copy the associated JDBC driver to the Kafka Connect Container. For the Debezium connectors, the location where you copy the driver JAR is fixed. Additionally, the Debezium connectors require that the JAR file is deployed within the container using a specific file name.

Find out the name or ID of the Kafka Connect container. You will need to pass the container name or ID as a parameter in some of the commands you will be running.The container name and ID can be listed using docker ps. For example:
docker ps -a --format '{{.ID}}\t{{.Names}}' --filter "name=kafka-connect"

The Kafka Connect container will either be called cspce-kafka-connect-1 or cspce_kafka-connect_1.

Steps
Debezium Db2 Source
  1. Download the Db2 JDBC driver.

    The JDBC driver can be downloaded from Maven Central. Cloudera recommends that you use version 11.5.0.0.

  2. Navigate to the directory where the driver was extracted.
  3. Copy the driver to the Kafka Connect container and rename it.
    docker cp ./jcc-[***VERSION***].jar [***KAFKA CONNECT CONTAINER NAME OR ID***]:/opt/connect/plugin/libs/debezium-connector-db2/db2-connector-java.jar
  4. Log in to the Kafka Connect container as the root user.
    docker exec -it --user root [***KAFKA CONNECT CONTAINER ID OR NAME***] /bin/bash
  5. Change the owner of the driver to the kafka user.
    chown kafka:cloudera /opt/connect/plugin/libs/debezium-connector-db2/db2-connector-java.jar
Debezium MySQL Source
  1. Download the MySQL JDBC driver.

    The JDBC driver can be downloaded from the MySQL Community Downloads page. Download the platform independent version. Cloudera recommends that you use version 8.0.27.

  2. Extract the files from the downloaded archive.
    You can use the tar command or any other archive manager to extract the archive. For example:
    tar -xf mysql-connector-java-[***VERSION***].tar.gz
  3. Navigate to the directory where the driver was extracted.
    cd mysql-connector-java-[***VERSION***]
  4. Copy the driver to the Kafka Connect container and rename it.
    docker cp ./mysql-connector-java-[***VERSION***].jar [***KAFKA CONNECT CONTAINER NAME OR ID***]:/opt/connect/plugin/libs/debezium-connector-mysql/mysql-connector-java.jar
  5. Log in to the Kafka Connect container as the root user.
    docker exec -it --user root [***KAFKA CONNECT CONTAINER ID OR NAME***] /bin/bash
  6. Change the owner of the driver to the kafka user.
    chown kafka:cloudera /opt/connect/plugin/libs/debezium-connector-mysql/mysql-connector-java.jar
Debezium Oracle Source
  1. Download the Oracle JDBC driver.

    The JDBC driver can be downloaded from Maven Central. Cloudera recommends that you use version 12.2.0.1, 19.8.0.0, or 21.1.0.0.

  2. Navigate to the directory where the driver was downloaded.
  3. Copy the driver to the Kafka Connect container and rename it.
    docker cp ./ojdbc8-[***VERSION***].jar [***KAFKA CONNECT CONTAINER NAME OR ID***]:opt/connect/plugin/libs/debezium-connector-oracle/oracle-connector-java.jar
  4. Log in to the Kafka Connect container as the root user.
    docker exec -it --user root [***KAFKA CONNECT CONTAINER ID OR NAME***] /bin/bash
  5. Change the owner of the driver to the kafka user.
    chown kafka:cloudera /opt/connect/plugin/libs/debezium-connector-oracle/oracle-connector-java.jar
Debezium SQL Server source
  1. Download the SQL Server JDBC driver.

    The JDBC driver can be downloaded from the Microsoft JDBC Driver for SQL Server downloads page. Cloudera recommends that you use version 7.2.2.jre8. Download the zipped tar file, do not download the executable version.

  2. Extract the files from the downloaded archive.
    You can use the tar command or any other archive manager to extract the archive. For example:
    tar -xf sqljdbc_[***VERSION***]_enu.tar.gz
  3. Navigate to the directory where the driver was extracted.
    cd sqljdbc_[***VERSION***]/enu
  4. Copy the driver to the Kafka Connect container and rename it.
    docker cp ./mssql-jdbc-[***VERSION***].jre8.jar [***KAFKA CONNECT CONTAINER NAME OR ID***]:/opt/connect/plugin/libs/debezium-connector-sqlserver/sqlserver-connector-java.jar
  5. Log in to the Kafka Connect container as the root user.
    docker exec -it --user root [***KAFKA CONNECT CONTAINER ID OR NAME***] /bin/bash
  6. Change the owner of the driver to the kafka user.
    chown kafka:cloudera /opt/connect/plugin/libs/debezium-connector-sqlserver/sqlserver-connector-java.jar
JDBC Source and Sink
  1. Download the driver.

    The driver you need to download will depend on your use case and how you plan on using the JDBC Source or Sink connector. For example. if you want the connector to connect to a MySQL database, you need to download and deploy the MySQL JDBC driver.

  2. Extract the files from the downloaded archive. You can use the tar command or any other archive manager to extract the archive. For example:
    tar -xf [***PATH TO JDBC DRIVER ARCHIVE***]
    Depending on the driver you downloaded, this step might not be required.
  3. Copy the driver to the Kafka connect container. For example:
    docker cp [***PATH TO JDBC DRIVER JAR***] [***KAFKA CONNECT CONTAINER NAME OR ID***]:/opt/connect/plugin/libs/

    You can copy the driver to any location within the container. Cloudera recommends that you copy it to a directory that has no other JAR files. Additionally, ensure that you take note of the path as you must provide it in the connector's configuration during connector deployment.

  4. Change the owner of the driver to the kafka user. 
    chown kafka:cloudera [***PATH TO DRIVER JAR***]
The selected drivers are copied within the Kafka Connect container. Deploying the Debezium, JDBC Source, and JDBC Sink connectors is now possible.
Deploy your selected connector. For more information on how to deploy the connectors, their configuration, and features, see the Related Information section at the bottom of this page.