6. Using Cache Pools and Directives

You can use the Command-Line Interface (CLI) to create, modify, and list Cache Pools and Cache Directives via the hdfs cacheadmin subcommand.

Cache Directives are identified by a unique, non-repeating, 64-bit integer ID. IDs will not be reused even if a Cache Directive is removed.

Cache Pools are identified by a unique string name.

You must first create a Cache Pool, and then add Cache Directives to the Cache Pool.

Cache Pool Commands

  • addPool -- Adds a new Cache Pool.

    Usage:

    hdfs cacheadmin -addPool <name> [-owner <owner>] [-group <group>] 
    [-mode <mode>] [-limit <limit>] [-maxTtl <maxTtl>] 

    Options:

     

    Table 3.1. Cache Pool Add Options

    OptionDescription
    <name>The name of the pool.
    <owner>The user name of the owner of the pool. Defaults to the current user.
    <group>The group that the pool is assigned to. Defaults to the primary group name of the current user.
    <mode>The UNIX-style permissions assigned to the pool. Permissions are specified in octal (e.g. 0755). Pool permissions are set to 0755 by default.
    <limit>The maximum number of bytes that can be cached by directives in the pool, in aggregate. By default, no limit is set.
    <maxTtl>The maximum allowed time-to-live for directives being added to the pool. This can be specified in seconds, minutes, hours, and days (e.g. 120s, 30m, 4h, 2d). Valid units are [smhd]. By default, no maximum is set. A value of "never" specifies that there is no limit.


  • modifyPool -- Modifies the metadata of an existing Cache Pool.

    Usage:

    hdfs cacheadmin -modifyPool <name> [-owner <owner>] [-group <group>] 
    [-mode <mode>] [-limit <limit>] [-maxTtl <maxTtl>] 

    Options:

     

    Table 3.2. Cache Pool Modify Options

    OptionDescription
    <name>The name of the pool to modify.
    <owner>The user name of the owner of the pool.
    <group>The group that the pool is assigned to.
    <mode>The UNIX-style permissions assigned to the pool. Permissions are specified in octal (e.g. 0755).
    <limit>The maximum number of bytes that can be cached by directives in the pool, in aggregate.
    <maxTtl>The maximum allowed time-to-live for directives being added to the pool. This can be specified in seconds, minutes, hours, and days (e.g. 120s, 30m, 4h, 2d). Valid units are [smdh]. By default, no maximum is set. A value of "never" specifies that there is no limit.


  • removePool -- Removes a Cache Pool. This command also "un-caches" paths that are associated with the pool.

    Usage:

    hdfs cacheadmin -removePool <name> 

    Options:

     

    Table 3.3. Cache Pool Remove Options

    OptionDescription
    <name>The name of the Cache Pool to remove.


  • listPools -- Displays information about one or more Cache Pools, such as name, owner, group, permissions, and so on.

    Usage:

    hdfs cacheadmin -listPools [-stats] [<name>] 

    Options:

     

    Table 3.4. Cache Pools List Options

    OptionDescription
    -statsDisplays additional Cache Pool statistics.
    <name>If specified, lists only the named Cache Pool.


  • help -- Displays detailed information about a command.

    Usage:

    hdfs cacheadmin -help <command-name> 

    Options:

     

    Table 3.5. Cache Pool Help Options

    OptionDescription
    <command-name>Displays detailed information for the specified command name. If no command name is specified, detailed help is displayed for all commands.


Cache Directive Commands

  • addDirective -- Adds a new Cache Directive.

    Usage:

    hdfs cacheadmin -addDirective -path <path> -pool <pool-name> [-force] 
    [-replication <replication>] [-ttl <time-to-live>] 

    Options:

     

    Table 3.6. Cache Pool Add Directive Options

    OptionDescription
    <path>The path to the cache directory or file.
    <pool-name>The Cache Pool to which the Cache Directive will be added. You must have Write permission for the Cache Pool in order to add new directives.
    -forceSkips checking of the Cache Pool resource limits.
    <replication>The UNIX-style permissions assigned to the pool. Permissions are specified in octal (e.g. 0755). Pool permissions are set to 0755 by default.
    <limit>The cache replication factor to use. Default setting is 1.
    <time-to-live>How long the directive is valid. This can be specified in minutes, hours and days (e.g. 30m, 4h, 2d). Valid units are [smdh]. A value of "never" indicates a directive that never expires. If unspecified, the directive never expires.


  • removeDirective -- Removes a Cache Directive.

    Usage:

    hdfs cacheadmin -removeDirective <id> 

    Options:

     

    Table 3.7. Cache Pools Remove Directive Options

    OptionDescription
    <id>The ID of the Cache Directive to remove. You must have Write permission for the pool that the directive belongs to in order to remove it. You can use the -listDirectives command to display a list of Cache Directive IDs.


  • removeDirectives -- Removes all of the Cache Directives in a specified path.

    Usage:

    hdfs cacheadmin -removeDirectives <path> 

    Options:

     

    Table 3.8. Cache Pool Remove Directives Options

    OptionDescription
    <path>The path of the Cache Directives to remove. You must have Write permission for the pool that the directives belong to in order to remove them. You can use the -listDirectives command to display a list of Cache Directives.


  • listDirectives -- Returns a list of Cache Directives.

    Usage:

    hdfs cacheadmin -listDirectives [-stats] [-path <path>] [-pool <pool>] 

    Options:

     

    Table 3.9. Cache Pools List Directives Options

    OptionDescription
    -statsLists path-based Cache Directive statistics.
    <path>Lists only the Cache Directives in the specified path. If there is a Cache Directive in the <path> that belongs to a Cache Pool for which you do not have Read access, it will not be listed.
    <pool>Lists on the Cache Directives in the specified Cache Pool.