Kudu tablet server metrics and table-level relabeling

Learn how to normalize Kudu tablet server metrics to enable table-level queries in Prometheus by using metric relabeling and the json_exporter tool.

Metric normalization overview

The Kudu Prometheus endpoint (/metrics_prometheus) embeds the tablet ID directly into each metric name, such as kudu_tablet_<tablet-id>_rows_upserted . Because Prometheus requires stable metric names with labels for dimensions, you must use two additional components to make metrics queryable by table name:

  • Metric relabeling: You must configure the Prometheus scrape configuration to extract the tablet ID into a tablet_id label and normalize the metric name.
  • json_exporter: This component reads the Kudu JSON metrics endpoint and exposes a kudu_tablet_info metric. This metric maps each tablet_id to its corresponding table_name, allowing dashboard queries to join on the ID and filter by table.

Configuring json_exporter

To set up the mapping between tablet IDs and table names, you must install and configure the json_exporter tool.
  1. Download the latest release from the prometheus-community/json_exporter page.
  2. Place the binary file on a host that the Prometheus server can reach.
  3. Create a configuration file named kudu_tablet_info.yml with the following content:
    
    modules:
      default:
        metrics:
          - name: kudu_tablet
            type: object
            path: "{[?(@.type=='tablet')]}"
            labels:
              tablet_id: "{.id}"
              table_name: "{.attributes.table_name}"
              table_id: "{.attributes.table_id}"
            values:
              info: "1"
  4. Start the json_exporter tool by running the following command:
    ./json_exporter --config.file=kudu_tablet_info.yml --web.listen-address=:7979