Cloudera Docs

Configuring exporter

You can deploy a YAML file containing telemetry export and exporter configurations to transmit logs and metrics from Cloudera Data Servicesenvironment to external destinations. You can also create and deploy credential secrets to the Observability namespace to authorize this configuration, and run a validation script to confirm your OpenTelemetry pipeline connections.

Workflow checklist

Prepare the required components before starting the configuration process:

Workflow component Description
Verify Cloudera Data Services on premises version Confirm your cluster runs Cloudera Data Services on premises 1.5.5 SP3 and higher.
Verify Cloudera Data Services on cloud version Cloudera AI on cloud 2.0.58 and higher created or upgraded after June 30, 2026
Prepare configuration Create a my-export.yaml patch file matching your target endpoint environment.
Prepare credentials Organize authentication credentials for your Kubernetes secret setup.
Download script Download the configure-observability-pipeline.sh validation script.
Cloudera Data Engineering 1.5.5 SP4 Enable Third-Party Export check box after the configuration is complete. For more information, see Adding a Cloudera Data Engineering service step 9.
Cloudera Data Warehouse 1.5.5 SP4 Enable OpenTelemetry export check box after the configuration is complete. For more information, see Enabling OpenTelemetry export in Cloudera Data Warehouse
  1. Create Your telemetry export patch file.
    To establish how your cluster metrics and logs are delivered, define the target endpoints in a patch file. Create a minimal my-export.yaml file by running the following command cat <<EOF > my-export.yaml You can use the below example as a reference: 
    spec:
  telemetryExport:
    enabled: true
    metrics:
      exporters:
        - type: file
          name: metrics
          config:
            path: /tmp/telemetry-export/metrics.jsonl
            format: json
            create_directory: true
    logs:
      exporters:
        - type: file
          name: logfile
          config:
            path: /tmp/telemetry-export/logs.jsonl
            format: json
            create_directory: true
  2. Create and deploy your secret credentials.
    Exporters require authentication and authorization details. You must create and apply a generic Kubernetes secret to securely transmit these details to the Observability namespace.

    Secret reference:

    This reference configuration helps you to understand the format in which you must base your secrets for the exporter. The generic end point format is ${secret:[***secret-name***]:[***secret-key***]}

    spec:
  telemetryExport:
    enabled: true
    metrics:
      exporters:
        - type: prometheusremotewrite
          name: thanosdev
          config:
            endpoint: ${secret:thanos-dev-credentials:metrics-ingestion-url}
            timeout: 30s
            retry_on_failure:
              enabled: true
              initial_interval: 10s
              max_interval: 40s
              max_elapsed_time: 300s
            target_info:
              enabled: false
            auth:
              authenticator: cdpauth/thanosdev
      extensions:
        - type: cdpauth
          name: thanosdev
          config:
            cdp_access_key_id: ${secret:thanos-dev-credentials:access-key-id}
            send_token: true
            token_valid_for_sec: 3600
            cdp_private_key: ${secret:thanos-dev-credentials:private-key}
            cdp_authentication_method: ${secret:thanos-dev-credentials:authentication-method}
      processors:
        - type: filter
          name: ksm-only
          config:
            error_mode: ignore
            metrics:
              datapoint:
                - resource.attributes["service.name"] != "kube-state-metrics"
  3. There are two configuration options.
    • If you are on Cloudera Data Services on premises, download the configure-observability-pipeline.sh script from the Download scripts documentation and run the validation script:

      The configure-observability-pipeline.sh script modifies your environment by applying configuration changes directly against your custom resources and validating connections to identify errors or review pod restarts. Download and run the validation script from your command line: my-export.yaml

      Configuration examples:

      The below are configuration examples. Provide your parameters at the command prompt to configure telemetry export configuration:
      1. Apply a telemetry export configuration: configure-observability-pipeline.sh --controlplane-namespace cdp --export-patch my-export.yaml
      2. Disable telemetry export: configure-observability-pipeline.sh --controlplane-namespace cdp --export-disable
    • If you are on Cloudera Data Services on cloud, you can run the following command to apply or disable telemetry export configuration:
      • Apply a telemetry export configuration: kubectl patch observabilitypipeline observability-pipeline -n monitoring --type merge --patch-file my-export.yaml
      • Disable telemetry export: kubectl patch ObservabilityPipeline observability-pipeline --type=merge -p '{"spec": {"telemetryExport": {"enabled": '"false"'}}}'
  4. If you want to export from Cloudera Data Engineering, you must enable the third-party export check box in Cloudera Data Engineering. For more information, see Adding a Cloudera Data Engineering service step 9
  5. If you want to export from Cloudera Data Warehouse, you must enable third-party export check box in Cloudera Data Warehouse. For more information, see Enabling OpenTelemetry export in Cloudera Data Warehouse