Updating Flink job dependencies

When you migrate your Flink jobs to a cluster that has a new supported version of Flink, the applications need to use a new version of the artifacts provided by the Flink deployment in your cluster. To avoid incompabilities between the packaged artifacts of your application and the artifacts provided by the Flink cluster, ensure that the POM file of the application is updated to match the Flink version of the new CDP Private Cloud Basecluster.

  1. Navigate to Management Console > Environments, and select the environment where you have created your cluster.
  2. Select the Streaming Analytics cluster from the list of Data Hub clusters.
  3. Access the latest Flink version of your cluster.
    1. Go to your cluster in Cloudera Manager.
    2. Select Hosts > Parcels from the main menu of Cloudera Manager.
    3. Search for Flink under Parcel Name, and review the Flink version.

      Name: FLINK

      Version: 1.13.2-csadh1.5.0.0-cdh7.2.12.0-35-17794544

  4. Update the <flink.version> property of your POM file using the Flink and Cloudera Streaming Analyitcs (CSA) version of your Data Hub cluster.
    You need to copy and paste only the prefix before the 'cdh' version number: 1.13.2-csadh1.5.0.0.
    <properties>
            ...
            <flink.version>1.13.2-csadh1.5.0.0</flink.version>
            ...
        </properties>
        ...
        <dependencies>
            ...
            <dependency>
                <groupId>org.apache.flink</groupId>
                <artifactId>flink-java</artifactId>
                <version>${flink.version}</version>
            </dependency>
            ...
        <dependencies>
    
  5. Rebuild your JAR file.
    mvn clean package