Cloudera Docs

Viewing logs for Cloudera Lakehouse Optimizer

You can view the service log files and event logs for Cloudera Lakehouse Optimizer.

  • Event logs
    By default, the Cloudera Lakehouse Optimizer Data Hub contains a read-only Iceberg table named sys.task_events in the default namespace to host the task metadata or event logs which enables historical trend analysis through queries. Cloudera Lakehouse Optimizer automatically logs the maintenance task metadata into this table every 20 minutes. You can query the table for root cause analysis, troubleshooting, and reporting purposes.
    You can perform the following operations related to this table:

    • Configure the ingestion schedule in the Cloudera Manager > Clusters > cloudera_lakehouse_optimizer > Configuration > dlm.task.cleanup.and.ingestion.interval.seconds advanced configuration snippet.

    • Change the namespace from sys to another namespace in the Cloudera Manager > Clusters > cloudera_lakehouse_optimizer > Configuration > conf/dlm-client.properties_role_safety_valve > dlm.event.log.namespace=[***ENTER NAMESPACE NAME***] property.

    • Change the table name from task_events to another table name in the Cloudera Manager > Clusters > cloudera_lakehouse_optimizer > Configuration > dlm.task.event.log.iceberg.table advanced configuration snippet.

    • Run queries on the table. For example, the select * from sys.task_events where tablename='demo_table'; query fetches the event log details for the demo_table Iceberg table in the default namespace; the select count(*) as maintenance_tasks, [***TABLE NAME***] from sys.task_events group by [***TABLE NAME***]; query fetches the number of maintenance tasks that ran on the specified table.

  • Service log files
    The service log files are available in Cloudera Manager. Perform the following steps to view the service logs.
    1. Click CLO Server on the Cloudera Manager > Clusters > cloudera_lakehouse_optimizer > Instances page.
    2. Click the Log Files drop-down, and then click Role Log File to view the complete service log.