You can forward logs from environments activated in Cloudera Data Warehouse (CDW) to
observability and monitoring systems such as Datadog, New Relic, or Splunk. You learn how to
configure a CDW environment for these systems.
After configuring log forwarding as described in this task, logs flow from CDW to your
system automatically. You enjoy the convenience of sorting, searching, and viewing logs on
your own system instead of grepping logs from diagnostic bundles on S3 or ABFS. In addition to
configuring the log forwarding, you configure removal of debug logs and text strings from the
logs. You can configure log forwarding to one of the following observability systems:
You create the log forwarding configuration in valid fluentd format. The configuration
is inserted into a larger fluentd configuration. All fluentd events are copied and relabeled
with the new label @cloudera_cdw. Your custom configuration is then inserted between
<label> tags:
<label @cloudera_cdw>
customer config goes
here
</label>
You can use any of the built-in fluentd filter, formatter, parser, or output plugins to build the custom config.
You must be familiar
with fluentd and accept the responsibility of configuring log forwarding to your
observability systems.
In the Data Warehouse service, go to the Environments tab.
Locate your environment, and click > Edit > Observability.
Decide how you want to create the fluentd config.
Write your own fluentd config from the ground up.
Use a Cloudera-provided snippet as a template to write your fluentd config.
In Log Forwarding Configuration, click .
Select one of the systems, such as Datadog, to configure.
A fluentd snippet appears. For example, the Datadog snippet appears:
Replace the snippet with the fluentd config you wrote from the ground up, or customize
the provided snippet.
For example, to customize the provided snippet replace the placeholder {{API Key}}
with the actual key.
(Optional) If debug level log messages are not designed, add a fluentd filter to remove
them: In the environment, click , and select Remove Debug Logs.
The fluentd snippet appears for removing debug logs. For example:
No user customization is necessary to remove debug logs.
(Optional) If certain log messages do not provide value for you, remove them with a
fluentd grep exclude filter: In the environment, click , select Grep
Exclude, and replace {{PATTERN}} with the grep expression that matches the
phrase you want to exclude.
If you use a proxy server for outbound traffic, provide the proxy server's CA
certificates PEM bundle as described in the next task.
Click Apply Changes.
CDW tests the log forwarding configuration and proxy CA certificates bundle, and
saves the configuration if both are valid. An invalid log forwarding
config error message appears in the event of a configuration problem. For
example:
If your configuration is valid, CDW initiates a restart of fluentd to apply the
updated config. You see the following indicators of success:
The environment Running indicator changes, blinks
Updating, and then once again says Running.
You see logs appearing in your observability system.
Many factors affect how
long it takes for forwarding to begin, but generally, the bigger your CDW
environment, the longer it takes.