Multiple exporter
The configuration that you must provide when you have multiple exporter to export the data.
Configuration
You can use telemetry export to configure advanced OpenTelemetry (OTel) pipelines, not just exporters. The following example shows how to connect all supported components (processors, extensions, exporters, and connectors) and inject them into the default observability pipeline:
The connection points are the user-defined connectors. In this example, these connectors are routing/metrics-fanout and routing/logs-fanout. These specific names are not required. The system recognizes and uses any user-defined routing connector. These user-defined routing connectors are the only connection points between the default OTel configuration and the export configuration. All other components (exporters, processors, extensions, and custom pipelines) are self-contained within the export configuration.
Although the example shows a complex setup, you only need a single exporter for an export configuration. The system automatically generates all other components if a user-defined version does not exist.
If no user-defined connector is present, the system auto-generates a routing connector and connects all user-defined pipelines to it. If no user-defined pipeline is present, the system also auto-generates the pipeline and automatically uses all user-defined processors and exporters.
spec:
telemetryExport:
enabled: true
# -------------------------------------------------------------------------
# METRICS
# -------------------------------------------------------------------------
metrics:
# --- Processors --------------------------------------------------------
processors:
- type: batch
name: metrics-batch
config:
send_batch_size: 500
send_batch_max_size: 1000
timeout: 10s
# --- Exporters ---------------------------------------------------------
exporters:
- type: file
name: cadvisor
config:
path: /tmp/telemetry-export/metrics-service-kubelet-cadvisor.jsonl
format: json
flush_interval: 5s
create_directory: true
- type: file
name: ksm
config:
path: /tmp/telemetry-export/metrics-service-kube-state-metrics.jsonl
format: json
flush_interval: 5s
create_directory: true
- type: file
name: metrics-other
config:
path: /tmp/telemetry-export/metrics-service-other.jsonl
format: json
flush_interval: 10s
create_directory: true
# --- Extensions --------------------------------------------------------
extensions:
# Exposes GET /health/status on port 13133
- type: health_check
name: export-health
config:
endpoint: "0.0.0.0:13133"
path: "/health/status"
check_collector_pipeline:
enabled: true
interval: 5m
exporter_failure_threshold: 5
# --- Connectors --------------------------------------------------------
connectors:
# Route by service.name — set by Prometheus as a resource attribute for each
# scrape target. Accessible in the routing connector's resource context via
# attributes["service.name"] (no "resource." prefix needed here).
# Note: the Prometheus "job" label ends up as a datapoint attribute, not a
# resource attribute, so attributes["job"] does not work for routing.
# Unmatched services fall through to metrics/other via default_pipelines.
- type: routing
name: metrics-fanout
config:
default_pipelines:
- metrics/other
error_mode: ignore
table:
- condition: attributes["service.name"] == "kubelet-cadvisor"
pipelines:
- metrics/cadvisor
- condition: attributes["service.name"] == "kube-state-metrics"
pipelines:
- metrics/ksm
# --- Pipelines ---------------------------------------------------------
pipelines:
- name: metrics/cadvisor
processors:
- batch/metrics-batch
exporters:
- file/cadvisor
- name: metrics/ksm
processors:
- batch/metrics-batch
exporters:
- file/ksm
- name: metrics/other
processors:
- batch/metrics-batch
exporters:
- file/metrics-other
# -------------------------------------------------------------------------
# LOGS
# -------------------------------------------------------------------------
logs:
# --- Processors --------------------------------------------------------
processors:
# Three filters, one per pipeline — each drops everything outside its own severity bucket.
- type: filter
name: warn-only
config:
error_mode: ignore
logs:
log_record:
- severity_number < SEVERITY_NUMBER_WARN or severity_number >= SEVERITY_NUMBER_ERROR
- type: filter
name: info-only
config:
error_mode: ignore
logs:
log_record:
- severity_number < SEVERITY_NUMBER_INFO or severity_number >= SEVERITY_NUMBER_WARN
- type: filter
name: other-only
config:
error_mode: ignore
logs:
log_record:
- severity_number >= SEVERITY_NUMBER_INFO
- type: batch
name: logs-batch
config:
send_batch_size: 1000
send_batch_max_size: 2000
timeout: 15s
# --- Exporters ---------------------------------------------------------
exporters:
- type: file
name: warn
config:
path: /tmp/telemetry-export/logs-level-warn.jsonl
format: json
flush_interval: 5s
create_directory: true
- type: file
name: info
config:
path: /tmp/telemetry-export/logs-level-info.jsonl
format: json
flush_interval: 5s
create_directory: true
- type: file
name: logs-other
config:
path: /tmp/telemetry-export/logs-level-other.jsonl
format: json
flush_interval: 5s
create_directory: true
# --- Extensions --------------------------------------------------------
extensions:
# Exposes Go runtime profiling on port 1777
- type: pprof
name: export-pprof
config:
endpoint: "0.0.0.0:1777"
# --- Connectors --------------------------------------------------------
connectors:
# Routing runs in the resource context where severity_number is not accessible,
# so all records are fanned out unconditionally to all three pipelines via a
# catch-all condition: "true". Each pipeline then filters to its own severity bucket.
- type: routing
name: logs-fanout
config:
error_mode: ignore
table:
- condition: "true"
pipelines:
- logs/warn
- logs/info
- logs/other
# --- Pipelines ---------------------------------------------------------
pipelines:
- name: logs/warn
processors:
- filter/warn-only
- batch/logs-batch
exporters:
- file/warn
- name: logs/info
processors:
- filter/info-only
- batch/logs-batch
exporters:
- file/info
- name: logs/other
processors:
- filter/other-only
- batch/logs-batch
exporters:
- file/logs-other
