Known issues and limitations

Stuck session jobs in Cloudera Streaming Analytics - Kubernetes Operator

Session jobs stop running if the session cluster's Job Manager is restarted without High Availability configured. However, because of a Flink bug, such stopped jobs get stuck in RECONCILING/STABLE state and cannot be restarted or deleted.

In such cases, the following is seen when using the kubectl get FlinkSessionJobs -n flink command:

kubectl get FlinkSessionJobs -n flink
NAME                         JOB STATUS    LIFECYCLE STATE
ssb-ssbdefault-testjobname   RECONCILING   STABLE

1. Remove the FlinkDeployment resource related to SSB:

   kubectl -n [***NAMESPACE***] delete flinkdeployment [***DEPLOYMENT NAME***]

2. Open the SSB configurations to edit:

   kubectl -n [***NAMESPACE***] edit cm ssb-config

3. Add the highlighted line to the security-context.yaml entry:

   allowPrivilegeEscalation: false
     capabilities:
       drop:
       - ALL
     runAsNonRoot: true
     seccompProfile:
       type: RuntimeDefault
     runAsUser: 9999

   It can take a couple of minutes for the changes to take effect after modifying the ConfigMap.

4. Execute the SQL job from Cloudera SQL Stream Builder again.

FLINK-33536: S3 filesystem sink and CSV format throws error

When using the Flink Table API CSV streaming sink with the S3 filesystem, the operation fails with IOException: Stream closed.

Cloudera SQL Stream Builder🔗

CSA-5622 [ssb][k8s] Project sync doesn't sync Kubernetes config for jobs

CSA-4858 - Kerberos encryption type detection does not always work correctly for Cloudera SQL Stream Builder

Cloudera SQL Stream Builder detects no supported encryption types even though there is a list of allowed encryption types in the krb5.conf file. This causes an error when generating keytabs from the principal and password pair.

1. Run ktutil on your cluster.

2. Change the configuration with the following commands:

   addent -password -p [*** USERNAME ***] -k 1 -e aes256-cts
   wkt /tmp/new_keytab.keytab

3. Upload the new keytab on Streaming SQL Console.

Flink🔗

DataStream conversion limitations

• Converting between Tables and POJO DataStreams is currently not supported in Cloudera Streaming Analytics - Kubernetes Operator.
• Object arrays are not supported for Tuple conversion.
• The java.time class conversions for Tuple DataStreams are only supported by using explicit TypeInformation: LegacyInstantTypeInfo, LocalTimeTypeInfo.getInfoFor(LocalDate/LocalDateTime/LocalTime.class).
• Only java.sql.Timestamp is supported for rowtime conversion, java.time.LocalDateTime is not supported.

Schema Registry catalog limitations

• Currently, the Schema Registry catalog / format only supports reading messages with the latest enabled schema for any given Kafka topic at the time when the SQL query was compiled.
• No time-column and watermark support for Registry tables.
• No CREATE TABLE support. Schemas have to be registered directly in the SchemaRegistry to be accessible through the catalog.
• The catalog is read-only. It does not support table deletions or modifications.

• By default, it is assumed that Kafka message values contain the schema id as a prefix, because this is the default behavior for the SchemaRegistry Kafka producer format.

  To consume messages with schema written in the header, the following property must be set for the Registry client: store.schema.version.id.in.header: true.