Cloudera Docs

Dynamic user pool assignment in Cloudera AI

Cloudera AI uses a dynamic pool assignment mechanism to manage user resource quotas in Cloudera AI 1.5.5 SP3 and higher releases. When quota management is enabled, the system automatically provisions and assigns resource pools to users.

Instead of creating a dedicated, named pool for each user at login, Cloudera AI maintains a shared set of precreated resource pools and dynamically assigns them to users when workloads are created. This approach enables more efficient pool management while continuing to enforce user resource quotas automatically.

The pool assignment lifecycle is as follows:

  1. The user registers or logs in for the first time.

    The system queues an asynchronous request to create a new user pool. This pool is not yet mapped to any user — it is stored in an internal user_pools table as an available pool.

  2. The user creates the first workload, that session, job, application, or model, and so on.

    The system assigns an available pool from the user_pools table to the user. From this point onward, that pool belongs to the user permanently.

  3. Running subsequent workloads.

    All future workloads created by the user run within the same assigned pool. The assignment is permanent — a user keeps the same pool for the lifetime of the workspace.

Table 1. Key behavioural differences with dynamic user pool assignment in earlier releases
Behaviour Cloudera AI 1.5.5 SP2 and lower releases Cloudera AI 1.5.5 SP3 and higher releases
Pool creation timing

Synchronous

It is created on the user’s first workload creation, that is session, job, and so on.

Asynchronous

It is created before assignment.
Pool-to-user mapping

Static

The user namespace and the user ID follow a predictable, 1:1 naming convention.

Dynamic

The user namespace and the user ID do not follow a predictable, 1:1 naming convention.
Pool assignment trigger User login User's first workload creation
Assignment permanence Permanent Permanent
Visible quotas in UI One per active user One quota entry per registered user - This can create a large number of displayed entries.