Using the OpenTelemetry Collector with Cloudera SQL Stream Builder [Technical Preview]

The OpenTelemetry metrics reporter makes it easier and more efficient to aggregate Cloudera SQL Stream Builder jobs' metrics to a central OpenTelemetry-compatible service using open standards.

The following artifacts need to be downloaded and copied to the /opt/cloudera/parcels/FLINK/lib/flink/lib/ directory on all the cluster nodes to use this feature.

• opentelemetry-api-1.30.0.jar
• opentelemetry-context-1.30.0.jar
• opentelemetry-exporter-common-1.30.0.jar
• opentelemetry-exporter-otlp-1.30.0.jar
• opentelemetry-exporter-otlp-common-1.30.0.jar
• opentelemetry-exporter-sender-okhttp-1.30.0.jar
• opentelemetry-sdk-common-1.30.0.jar
• opentelemetry-sdk-metrics-1.30.0.jar
• opentelemetry-semconv-1.30.0-alpha.jar

Restart Cloudera SQL Stream Builder after the JAR files have been downloaded.

Add the following SET commands to the beginning of your job to start collecting metrics with OpenTelemetry:

set metrics.reporters=otel;
set metrics.reporter.otel.factory.class=org.apache.flink.metrics.otel.OpenTelemetryMetricReporterFactory;
set metrics.reporter.otel.exporter.endpoint=http://[*** FQDN OR IP ADDRESS ***]:[*** PORT ***];
set metrics.reporter.otel.service.name=[*** SERVICE NAME ***];
set metrics.reporter.otel.service.version=[*** SERVICE VERSION ***];

For more information on using the Flink OpenTelemetry Metrics Reporter, refer to the Apache Flink documentation.