Removing scratch directories
You need to know how to periodically clear scratch directories used by Apache Hive to prevent problems, such as failing jobs.
Scratch directories where Hive stores intermediate, or temporary, files accumulate too much data over time and overflow. You can configure Hive to remove scratch directories periodically and without user intervention. Using Cloudera Manager, you add the following properties as shown in the procedure:
- hive.start.cleanup.scratchdir
- Value: true
- hive.server2.clear.dangling.scratchdir
- Value: true
- hive.server2.clear.dangling.scratchdir.interval
- Example Value: 1800s
- In Cloudera Manager, click . Clusters > Hive on Tez > Configuration.
- Search for the Hive Service Advanced Configuration Snippet (Safety Valve) for hive-site.xml setting.
- In the Hive Service Advanced Configuration Snippet (Safety Valve) for hive-site.xml setting, click +.
- InName enter the property name and in value enter the value.