Kudu Security Overview

Kudu uses an internal PKI to issue X.509 certificates to servers in the cluster. Connections between peers who have both obtained certificates will use TLS for authentication. In such cases, neither peer needs to contact the Kerberos KDC.

X.509 certificates are only used for internal communication among Kudu servers, and between Kudu clients and servers. These certificates are never presented in a public facing protocol. By using internally-issued certificates, Kudu offers strong authentication which scales to huge clusters, and allows TLS encryption to be used without requiring you to manually deploy certificates on every node.

After authenticating to a secure cluster, the Kudu client will automatically request an authentication token from the Kudu master. An authentication token encapsulates the identity of the authenticated user and carries the Kudu master's RSA signature so that its authenticity can be verified. This token will be used to authenticate subsequent connections. By default, authentication tokens are only valid for seven days, so that even if a token were compromised, it cannot be used indefinitely. For the most part, authentication tokens should be completely transparent to users. By using authentication tokens, Kudu is able to take advantage of strong authentication, without paying the scalability cost of communicating with a central authority for every connection.

When used with distributed compute frameworks such as Apache Spark, authentication tokens can simplify configuration and improve security. For example, the Kudu Spark connector will automatically retrieve an authentication token during the planning stage, and distribute the token to tasks. This allows Spark to work against a secure Kudu cluster where only the planner node has Kerberos credentials.

Users running client Kudu applications must first run the kinit command to obtain a Kerberos ticket-granting ticket. For example:

kinit admin@EXAMPLE-REALM.COM

Once authenticated, you use the same client code to read from and write to Kudu servers with and without the Kerberos configuration.

Apache Sentry models tabular objects in the following hierarchy:

• Server: is indicated by the Kudu configuration flag --server_name. Everything stored in a Kudu cluster falls within the given "server"
• Database: is indicated as a prefix of table names with the format <database>.<table>
• Table: is a single Kudu table.
• Column: is a column within a Kudu table.

Each level of this hierarchy defines a scope on which the privileges can be granted. Privileges granted on a higher scope imply privileges on a lower scope. For example, if a user has a SELECT privilege on a database, then that user implicitly has the SELECT privileges on every table belonging to that database.

Additionally, there are three special actions recognized by Kudu: ALL, OWNER, and METADATA. If a user has the ALL or OWNER privileges on a given table, that user has all of the above privileges on the table. METADATA privilege is not an actual privilege per se, rather, it is a conceptual privilege with which Kudu models any privilege. If a user has any privilege on a given table, that user has METADATA privileges on the table, i.e. a privilege granted on any action on a table implies that the user has the METADATA privilege on that table.

For more details about Sentry privileges, see the Apache Sentry documentation.

When a Kudu master receives a request, it consults Sentry to determine what privileges a user has. If the user is not authorized to perform the requested action, the request is rejected. Kudu leverages the authenticated identity of a user to decide whether to perform or reject a request.

Rather than having every tablet server communicate directly with Sentry, privileges are propagated and checked via authorization tokens. These tokens encapsulate what privileges a user has on a given table. Tokens are generated by the master and returned to Kudu clients upon opening a Kudu table. Kudu clients automatically attach authorization tokens when sending requests to tablet servers.

Authorization tokens are a means to limit the number of nodes directly accessing Sentry to retrieve privileges. As such, since the expected number of tablet servers in a cluster is much higher than the number of Kudu masters, they are only used to authorize requests sent to tablet servers. Kudu masters fetch privileges directly from Sentry or cache. See <<privilege-caching>> for more details of Kudu's privilege cache.

Similar to the validity interval for authentication tokens, to limit the window of potential unwanted access if a token becomes compromised, authorization tokens are valid for five minutes by default. The acquisition and renewal of a token is hidden from the user, as Kudu clients automatically retrieve new tokens when existing tokens expire.

When a tablet server that has been configured to enforce fine-grained access control receives a request, it checks the privileges in the attached token, rejecting it if the privileges are not sufficient to perform the requested operation, or if it is invalid (e.g. expired).

It may be desirable to allow certain users to view and modify any data stored in Kudu. Such users can be specified via the --trusted_user_acl master configuration. Trusted users can perform any operation that would otherwise require fine-grained privileges, without Kudu consulting Sentry.

Additionally, some services that interact with Kudu may authorize requests on behalf of their end users. For example, Apache Impala authorizes queries on behalf of its users, and sends requests to Kudu as the Impala service user, commonly "impala". Since Impala authorizes requests on its own, to avoid extraneous communication between Sentry and Kudu, the Impala service user should be listed as a trusted user.

Sentry is often configured with Kerberos authentication. In order to enable integration with Sentry, a cluster must first be integrated with the Apache Hive Metastore. See the Enabling the Hive Metastore Integration section to configure Kudu to synchronize its internal catalog with the Hive Metastore.

--sentry_service_rpc_addresses=<Sentry RPC address>
--server_name=<value of HiveServer2's hive.sentry.server configuration>
--kudu_service_name=kudu
--sentry_service_kerberos_principal=sentry
--sentry_service_security_mode=kerberos

# This example ACL setup allows the 'impala' user to access all data stored in
# Kudu, assuming Impala will authorize requests on its own. The 'hadoopadmin'
# user is also granted access to all Kudu data, which may facilitate testing
# and debugging.
--trusted_user_acl=impala,hadoopadmin

--tserver_enforce_access_control=true

The following authorization policy is enforced by Kudu masters:

The following authorization policy is enforced by Kudu tablet servers: