Run Hue Document Cleanup

If your cluster uses Hue, perform the following steps (not required for maintenance releases). These steps clean up the database tables used by Hue and can help improve performance after an upgrade.
  1. Back up your database before starting the cleanup activity.
  2. Check the saved documents such as Queries and Workflows for a few users to prevent data loss.
  3. Connect to the Hue database. See Hue Custom Databases in the Hue component guide for information about connecting to your Hue database.
  4. Check the size of the desktop_document, desktop_document2, oozie_job, beeswax_session, beeswax_savedquery and beeswax_queryhistory tables to have a reference point. If any of these have more than 100k rows, run the cleanup.
    select count(*) from desktop_document;
    select count(*) from desktop_document2;
    select count(*) from beeswax_session;
    select count(*) from beeswax_savedquery;
    select count(*) from beeswax_queryhistory;
    select count(*) from oozie_job;
  5. SSH in to an active Hue instance.
  6. If you are upgrading from CDH 5.x or 6.x to CDP, then follow the below steps:
    1. Download the script_runner directory by any of the following methods:
      git clone https://github.com/cloudera/hue/tree/master/tools/ops/script_runner /opt/cloudera/hue_scripts
    2. Run the script as the root user:
      DESKTOP_DEBUG=True /opt/cloudera/hue_scripts/script_runner hue_desktop_document_cleanup --keep-days 30

      The logs are displayed on the console because DESKTOP_DEBUG is set to True. Alternatively, you can view the logs from the following location: /var/log/hue/hue_desktop_document_cleanup.log. The first run can typically take around 1 minute per 1000 entries in each table.

    3. Check the size of the desktop_document, desktop_document2, oozie_job, beeswax_session, beeswax_savedquery and beeswax_queryhistory tables and confirm they are now smaller.
      select count(*) from desktop_document;
      select count(*) from desktop_document2;
      select count(*) from beeswax_session;
      select count(*) from beeswax_savedquery;
      select count(*) from beeswax_queryhistory;
      select count(*) from oozie_job;
    4. If the hue_scripts script has run successfully, the table size should decrease, and you can now set up a cron job for scheduled cleanups.

    5. Copy the wrapper script for cron by running the following command:
      cp /opt/cloudera/hue_scripts/hue_history_cron.sh /etc/cron.daily
    6. Specify the cleanup interval in the --keep-days property in the hue_history_cron.sh file as shown in the following example:
      ${SCRIPT_DIR}/script_runner hue_desktop_document_cleanup --keep-days 120

      In this case, the data will be retained in the tables for 120 days.

    7. Change the permissions on the script so only the root user can run it.
      chmod 700 /etc/cron.daily/hue_history_cron.sh
  7. If you are upgrading from a previous CDP release, the follow the below steps:
    1. Change to the Hue home directory:
      cd /opt/cloudera/parcels/CDH/lib/hue
    2. Run the following command as the root user:
      DESKTOP_DEBUG=True ./build/env/bin/hue desktop_document_cleanup --keep-days x

      The --keep-days property is used to specify the number of days for which Hue will retain the data in the backend database.

      For example:
      DESKTOP_DEBUG=True ./build/env/bin/hue desktop_document_cleanup --keep-days 90

      In this case, Hue will retain data for the last 90 days.

      The logs are displayed on the console because DESKTOP_DEBUG is set to True. Alternatively, you can view the logs from the following location:

      /var/log/hue/desktop_document_cleanup.log

      The first run can typically take around 1 minute per 1000 entries in each table, but can take much longer depending on the size of the tables.

    3. Check the size of the desktop_document, desktop_document2, oozie_job, beeswax_session, beeswax_savedquery and beeswax_queryhistory tables and confirm they are now smaller.
      select count(*) from desktop_document;
      select count(*) from desktop_document2;
      select count(*) from beeswax_session;
      select count(*) from beeswax_savedquery;
      select count(*) from beeswax_queryhistory;
      select count(*) from oozie_job;
    4. If any of the tables are still above 100k in size, run the command again while specifying less number of days this time. For example, 60 or 30.