Cloudera Docs

Set up a storage policy for HDFS

You can use Cloudera Manager to set up storage policy on a DataNode data directory.

  1. Check the HDFS Service Advanced Configuration Snippet (Safety Valve) for hdfs-site.xml to be sure that dfs.storage.policy.enabled has not been changed from its default value of true.
  2. Specify the storage types for each DataNode Data Directory that is not a standard disk, by adding the storage type in brackets at the beginning of the directory path.
    For example:
    [SSD]/dfs/dn1
[DISK]/dfs/dn2
[ARCHIVE]/dfs/dn3
  3. Open a terminal session on any HDFS host and run the hdfs storagepolicies -setStoragePolicy command for each path on which you want to set a storage policy. 
    hdfs storagepolicies -setStoragePolicy -path <path> -policy <policy>
path_to_file_or_directory -policy policy_name
  4. To move the data to the appropriate storage based on the current storage policy, use the mover utility, from any HDFS host.
    Use mover -h to get a list of available options. To migrate all data at once (this may take a long time), you can set the path to /.
    hdfs mover -p <path>