Scaling Namespaces and Optimizing Data Storage
Also available as:
PDF
loading table of contents...

Commands for using cache pools and directives

You can use the Command-Line Interface (CLI) to create, modify, and list cache pools and cache directives using the hdfs cacheadmin subcommand.

Cache Directives are identified by a unique, non-repeating, 64-bit integer ID. IDs will not be reused even if a Cache Directive is removed.

Cache Pools are identified by a unique string name.

You must first create a Cache Pool, and then add Cache Directives to the Cache Pool.

Cache Pool Commands

  • addPool -- Adds a new Cache Pool.

    Usage:

    hdfs cacheadmin -addPool <name> [-owner <owner>] [-group <group>] 
    [-mode <mode>] [-limit <limit>] [-maxTtl <maxTtl>]
    Options:
    Table 1. Cache Pool Add Options
    Option Description
    <name> The name of the pool.
    <owner> The user name of the owner of the pool. Defaults to the current user.
    <group> The group that the pool is assigned to. Defaults to the primary group name of the current user.
    <mode> The UNIX-style permissions assigned to the pool. Permissions are specified in octal (e.g. 0755). Pool permissions are set to 0755 by default.
    <limit> The maximum number of bytes that can be cached by directives in the pool, in aggregate. By default, no limit is set.
    <maxTtl> The maximum allowed time-to-live for directives being added to the pool. This can be specified in seconds, minutes, hours, and days (e.g. 120s, 30m, 4h, 2d). Valid units are [smhd]. By default, no maximum is set. A value of "never" specifies that there is no limit.
  • modifyPool -- Modifies the metadata of an existing Cache Pool.

    Usage:

    hdfs cacheadmin -modifyPool <name> [-owner <owner>] [-group <group>] 
    [-mode <mode>] [-limit <limit>] [-maxTtl <maxTtl>]
    Options:
    Table 2. Cache Pool Modify Options
    Option Description
    <name> The name of the pool to modify.
    <owner> The user name of the owner of the pool.
    <group> The group that the pool is assigned to.
    <mode> The UNIX-style permissions assigned to the pool. Permissions are specified in octal (e.g. 0755).
    <limit> The maximum number of bytes that can be cached by directives in the pool, in aggregate.
    <maxTtl> The maximum allowed time-to-live for directives being added to the pool. This can be specified in seconds, minutes, hours, and days (e.g. 120s, 30m, 4h, 2d). Valid units are [smdh]. By default, no maximum is set. A value of "never" specifies that there is no limit.
  • removePool -- Removes a Cache Pool. This command also "un-caches" paths that are associated with the pool.

    Usage:

    hdfs cacheadmin -removePool <name>
    Options:
    Table 3. Cache Pool Remove Options
    Option Description
    <name> The name of the Cache Pool to remove.
  • listPools -- Displays information about one or more Cache Pools, such as name, owner, group, permissions, and so on.

    Usage:

    hdfs cacheadmin -listPools [-stats] [<name>]
    Options:
    Table 4. Cache Pools List Options
    Option Description
    -stats Displays additional Cache Pool statistics.
    <name> If specified, lists only the named Cache Pool.
  • help -- Displays detailed information about a command.

    Usage:

    hdfs cacheadmin -help <command-name>
    Options:
    Table 5. Cache Pool Help Options
    Option Description
    <command-name Displays detailed information for the specified command name. If no command name is specified, detailed help is displayed for all commands.

Cache Directive Commands

  • addDirective -- Adds a new Cache Directive.

    Usage:

    hdfs cacheadmin -addDirective -path <path> -pool <pool-name> [-force] 
    [-replication <replication>] [-ttl <time-to-live>]
    Options:
    Table 6. Cache Pool Add Directive Options
    Option Description
    <path> The path to the cache directory or file.
    <pool-name> The Cache Pool to which the Cache Directive will be added. You must have Write permission for the Cache Pool in order to add new directives.
    -force Skips checking of the Cache Pool resource limits.
    <replication> The cache replication factor to use. Default setting is 1.
    <time-to-live> How long the directive is valid. This can be specified in minutes, hours and days (e.g. 30m, 4h, 2d). Valid units are [smdh]. A value of "never" indicates a directive that never expires. If unspecified, the directive never expires.
  • removeDirective -- Removes a Cache Directive.

    Usage:

    hdfs cacheadmin -removeDirective <id>
    Options:
    Table 7. Cache Pool Remove Directive Options
    Option Description
    <id> The ID of the Cache Directive to remove. You must have Write permission for the pool that the directive belongs to in order to remove it. You can use the -listDirectives command to display a list of Cache Directive IDs.
  • removeDirectives -- Removes all of the Cache Directives in a specified path.

    Usage:

    hdfs cacheadmin -removeDirectives <path>
    Options:
    Table 8. Cache Pool Remove Directives Options
    Option Description
    <path> The path of the Cache Directives to remove. You must have Write permission for the pool that the directives belong to in order to remove them. You can use the -listDirectives command to display a list of Cache Directives.
  • listDirectives -- Returns a list of Cache Directives.

    Usage:

    hdfs cacheadmin -listDirectives [-stats] [-path <path>] [-pool <pool>]
    Options:
    Table 9. Cache Pools List Directives Options
    Option Description
    <path> Lists only the Cache Directives in the specified path. If there is a Cache Directive in the <path> that belongs to a Cache Pool for which you do not have Read access, it will not be listed.
    <pool> Lists on the Cache Directives in the specified Cache Pool.
    -stats Lists path-based Cache Directive statistics.