Cloudera Docs

Enabling Flink DEBUG logging

You can review the log text files of the Flink jobs when an error is detected during the processes. When you set the log level of Flink to DEBUG, you can easily trace the log file for errors.

A log file is created for every Flink process that contains messages for the different events happening in the given process. You can use these log files to solve the errors and problems that can occur during Flink processes. You can access the Flink logs using the Flink Dashboard.

  1. Navigate to the Configuration page in Cloudera Manager.
    1. Navigate to Management Console > Environments, and select the environment where you have created your cluster.
    2. Select the Streaming Analytics cluster from the list of Data Hub clusters.
    3. Select Cloudera Manager from the services.
      You are redirected to the Cloudera Manager user interface.
    4. Click Clusters > Flink.
    5. Click Configuration.
  2. Search for Flink Client Advanced Configuration Snippet (Safety Valve) for flink-conf/log4j.properties configuration.
  3. Add the following parameters to the Safety Valve: 
    logger.flink.name = org.apache.flink
logger.flink.level = DEBUG
  4. Click Save Changes.
  5. Restart the Flink service with Action > Restart.
    After updating the Flink log configuration in Cloudera Manager, you need to access the YARN Resource Manager user interface from the Streaming Analytics cluster page.
  6. Access the YARN Resource Manager user interface to stop the YARN job of the Flink application.
    1. Navigate to Management Console > Environments, and select the environment where you have created your cluster.
    2. Select the Streaming Analytics cluster.
    3. Select Resource Manager from the list of Services.
      You are redirected to the Resource Manager user interface.
    4. Select Applications.
      The running Flink applications are displayed.
  7. Select the application you need to stop.
  8. Click Settings.
  9. Select Kill application.
    After stopping the Flink application, you can review the log file of the Flink job by accessing the Flink Dashboard from the Streaming Analytics cluster page.
  10. Navigate to Flink Dashboard and review the log level for the running job.
    1. Navigate to Management Console > Environments, and select the environment where you have created your cluster.
    2. Select the Streaming Analytics cluster.
    3. Click Flink Dashboard from the list of services.
  11. Select Task Managers from the main menu.
  12. Select the previously submitted job.
  13. Click Logs.