Diagnostic Data Collection

Cloudera Manager log files provide important information for Cloudera Support to use in analyzing problems or unexpected behavior with Cloudera Manager deployments or CDH clusters. Altus Director triggers the collection of diagnostic data for deployments and clusters it manages. This helps prevent situations where a failed cluster has been terminated but Cloudera Support has no diagnostic data or log files to help identify the cause of the failure. If you have a Cloudera Enterprise or Cloudera Enterprise Trial license, diagnostic data is collected and sent to Cloudera Support automatically on cluster bootstrap or update failure. By default, diagnostic data is also downloaded to the Altus Director instance.

If Cloudera Manager cannot collect diagnostic data, no information is sent to Cloudera Support, and the Cloudera Manager service logs are downloaded to Altus Director instead of the diagnostic data. The logs contain less information than the diagnostic data, but can still be useful to Cloudera Support for analyzing deployment and cluster behavior.

You can initiate diagnostic data collection manually through the Altus Director web UI or API. You can collect diagnostic data for an entire Cloudera Manager deployment or for a specific CDH cluster.

For more information on how diagnostic data collection works in Cloudera Manager, see the Cloudera Manager documentation page Sending Usage and Diagnostic Data to Cloudera.

Manual Collection of Diagnostic Data

You can manually trigger the collection of diagnostic data using either the Altus Director web UI or the Altus Director API.

Using the Web UI

To trigger diagnostic data collection, perform the following steps:
  1. Go to the Altus Director web UI page for the deployment or cluster.
  2. Click the down arrow on the dropdown list to the right of the deployment or cluster name.
  3. In the dropdown list, click Collect Diagnostic Data.

Altus Director makes an API call to the Cloudera Manager API collectDiagnosticData. If successful, Cloudera Manager sends the diagnostic data to Cloudera Support and, if the download diagnostic data property is set to true in the Altus Director application.properties file, also downloads a zip file containing the diagnostic data for the deployment or cluster to the Altus Director EC2 instance. If diagnostic data collection is unsuccessful, and the download diagnostic data property is set to true, Cloudera Manager downloads the Cloudera Manager service logs to Altus Director.

Manually Triggering Collection of Diagnostic Data at Cluster Termination

When you terminate a Cloudera Manager deployment or CDH cluster in the web UI, the screen for confirming the termination includes a button that triggers collection of diagnostic data:



Using the API

To manually trigger collection of diagnostic data for Cloudera Manager deployments, use the API at http://cloudera_director_ip:port_number/api-console/index.html#!/deployments/collectDiagnosticData.

To manually trigger collection of diagnostic data for CDH clusters, use the API at http://cloudera_director_ip:port_number/api-console/index.html#!/clusters/collectDiagnosticData.

Status for Data Collection

While diagnostic data collection is in progress, the status of the deployment or cluster changes from its current state to Updating: Collecting diagnostic data when you mouse-over the Status bar:


The cluster status is not actually updated; the updating message is displayed simply to inform that diagnostic data collection is in progress. Because diagnostic data collection does not change the status of the cluster, when the data collection is complete, the deployment or cluster status message reverts to what it was before diagnostic data collection began.

If you click View Log Status on the deployment or cluster screen, the Diagnostic Log Summary is displayed, showing information about the last diagnostic data collection:


If diagnostic data has never been collected for the deployment or cluster, the Diagnostic Log Status value is Not Collected and there is no link to open the Diagnostic Log Status screen.

Configuring Diagnostic Data Collection

By default, Cloudera Manager sends diagnostic data to Cloudera Support and to Altus Director. You can configure diagnostic data collection on Cloudera Manager and Altus Director using the procedures described in this section.

Configuring Upload of Diagnostic Data to Cloudera Support

The Cloudera Manager server property that determines whether diagnostic data is automatically sent to Cloudera Support has the display name Send Diagnostic Data to Cloudera Automatically and the API name phone_home. The default value for this property is true. To disable diagnostic data collection in Cloudera Manager, set this property to false. Set the property in Cloudera Manager by following these steps:
  1. In Cloudera Manager, click Administration > Settings.
  2. In the list of Filters in the lefthand pane, click Support.
  3. Click the checkbox for the property Send Diagnostic Data to Cloudera Automatically to toggle the setting between true and false.

For more information on the phone_home property, see the table in the Support section of Cloudera Manager Server Properties.

Configuring Download of Diagnostic Data to Altus Director

Several Altus Director server configuration properties affect the way diagnostic data is handled. You can set these properties in the application.properties file located at /etc/cloudera-director-server/ on the Altus Director instance, or at the command line.
  • lp.debug.collectDiagnosticDataOnFailure: Determines whether automatic collection of diagnostic data occurs for cluster bootstrap or update failures. The default value is true.
  • lp.debug.downloadDiagnosticData: Determines whether diagnostic data is downloaded to the Altus Director instance. The default value is true.
  • lp.remote.terminate.assumeYes: Determines whether Altus Director skips prompting the user to confirm termination when the terminate-remote command is invoked. If you set the property to true, termination proceeds even if diagnostic data collection has failed. The default setting is false.
  • lp.debug.diagnosticDataDownloadDirectory: Sets a nondefault path for the download of diagnostic data for deployments and clusters. The default location is /tmp. The directory where diagnostic data has been downloaded appears in the File Path field in the Diagnostic Log Summary.
  • lp.debug.createDiagnosticDataDownloadDirectory: Determines whether Altus Director creates the nondefault download directory specified in lp.debug.diagnosticDataDownloadDirectory if it does not exist. The default value is true.

For information about setting Altus Director properties by using the CLI or editing the application.properties file, see Setting Altus Director Properties.