Redaction capabilities for diagnostic data

Describes the resources that you can configure for redaction. Cloudera recommends enabling redaction even if you are not sending diagnostic data to Telemetry Publisher.

The diagnostic data collected by Telemetry Publisher may contain sensitive information in job configuration or log files. The following lists the data and resources that you can configure for redacting sensitive data before it is sent to Telemetry Publisher:

• Log and query redaction — You can redact information in logs and queries collected by Telemetry Publisher based on filters created with regular expressions.
• MapReduce job properties redaction — You can redact job configuration properties before they are stored in HDFS. Since Telemetry Publisher reads the job configuration files from HDFS, it only fetches redacted configuration information.
• Spark event and executor log redaction — The spark.redaction.regex configuration property is used to redact sensitive data from event and executor logs in your YARN service. When this configuration property is enabled, Telemetry Publisher sends only redaction data to Cloudera Observability. By default, this configuration property is enabled, but it can be overridden by using the advanced configuration snippet in Cloudera Manager or in the Spark application itself.