Configuring exporter

You can deploy a YAML file containing telemetry export and exporter configurations to transmit logs and metrics from Cloudera Data Services on premises to external destinations. You can also create and deploy credential secrets to the Observability namespace to authorize this configuration, and run a validation script to confirm your OpenTelemetry pipeline connections.

You must upgrade to or install Cloudera Data Services on premises 1.5.5 SP3.

Workflow checklist

Prepare the required components before starting the configuration process:

Workflow component	Description
Verify Data Services version	Confirm your cluster runs Cloudera Data Services on premises 1.5.5 SP3.
Prepare configuration	Create a my-export.yaml patch file matching your target endpoint environment.
Prepare credentials	Organize authentication credentials for your Kubernetes secret setup.
Acquire script	Download the configure-observability-pipeline.sh validation script.

Create Your telemetry export patch file.
To establish how your cluster metrics and logs are delivered, define the target endpoints in a patch file. Create a minimal my-export.yaml file by running the following command at your command prompt: cat <<EOF > my-export.yaml You can use the below example as a reference:
```
spec:
  telemetryExport:
    enabled: true
    metrics:
      exporters:
        - type: file
          name: metrics
          config:
            path: /tmp/telemetry-export/metrics.jsonl
            format: json
            create_directory: true
    logs:
      exporters:
        - type: file
          name: logfile
          config:
            path: /tmp/telemetry-export/logs.jsonl
            format: json
            create_directory: true
```
note
For multiple exporter configuration, you must create a patch file similar to the reference provided as an example in the Advanced exporter documentation.

Create and deploy your secret credentials.

Exporters require authentication and authorization details. You must create and apply a generic Kubernetes secret to securely transmit these details to the Observability namespace.

Secret reference:

This reference configuration helps you to understand the format in which you must base your secrets for the exporter.

spec:
  telemetryExport:
    enabled: true
    metrics:
      exporters:
        - type: prometheusremotewrite
          name: thanosdev
          config:
            endpoint: ${secret:thanos-dev-credentials:metrics-ingestion-url}
            timeout: 30s
            retry_on_failure:
              enabled: true
              initial_interval: 10s
              max_interval: 40s
              max_elapsed_time: 300s
            target_info:
              enabled: false
            auth:
              authenticator: cdpauth/thanosdev
      extensions:
        - type: cdpauth
          name: thanosdev
          config:
            cdp_access_key_id: ${secret:thanos-dev-credentials:access-key-id}
            send_token: true
            token_valid_for_sec: 3600
            cdp_private_key: ${secret:thanos-dev-credentials:private-key}
            cdp_authentication_method: ${secret:thanos-dev-credentials:authentication-method}
      processors:
        - type: filter
          name: ksm-only
          config:
            error_mode: ignore
            metrics:
              datapoint:
                - resource.attributes["service.name"] != "kube-state-metrics"

Download and run the validation script.
The configure-observability-pipeline.sh script modifies your environment by applying configuration changes directly against your custom resources and validating connections to identify errors or review pod restarts. Download and run the validation script from your command line: my-export.yaml

Configuration examples:
The below are configuration examples. Provide your parameters at the command prompt to configure telemetry export configuration:
1. Apply a telemetry export configuration: configure-observability-pipeline.sh --controlplane-namespace cdp --export-patch my-export.yaml
2. Disable telemetry export: configure-observability-pipeline.sh --controlplane-namespace cdp --export-disable