Cloudera Docs

Configuring logging in Cloudera Data Engineering jobs

You can configure logging in your Cloudera Data Engineering (CDE) jobs using the CDE UI. You can modify log levels, and add or replace customized log4j.properties files.

  1. In the Cloudera Data Platform (CDP) management console, click the Data Engineering tile and click Overview.
  2. In the CDE Services column, select the environment containing the virtual cluster that your application is associated with.
  3. In the Virtual Clusters column on the right, click the View Jobs icon on the virtual cluster containing the application you want to manage.
  4. Select Jobs from the left hand menu.
  5. Click on the name of the job you want to modify.
  6. The Run History tab lists the recent job executions for the application. Click the Configuration tab to display the job configuration.
  7. Click Edit to open the job configuration for editing.
  8. Click Add Configuration.
    • Enter extraJavaOptions in the config_key field.
    • Enter '-Dlog4j.configuration=file:/app/mount/[***LOG4J.PROPERTIES FILE***]' in the config_value field, replacing [***LOG4J.PROPERTIES FILE***] with the name of the custom log4j.properties file you want to reference.
    For example:
    '-Dlog4j.configuration=file:/app/mount/mylog4j.properties'
  9. Open Advanced Options to see additional parameters.
  10. Under Other Dependencies click Upload.
    Browse for the custom log4j.properties file you want to upload, then click Open.
  11. Scroll down to set Log Level.
    Select a value from the drop-down list.
  12. Click Update.