Prerequisites for Cloudera Lakehouse Optimizer REST API authentication (trusted proxy)

On Cloudera Base on premises, trusted-proxy configuration, identity forwarding, and a valid signing key are required before Cloudera Lakehouse Optimizer REST API calls succeed through Apache Knox.

The topic describes adding users in the dlm_admin group, generating a Knox token, and calling the API with a Bearer token. Ensure to the meet the following prerequisites to run API calls including the sample health check, failing which, might return a 403 Forbidden or 401 Unauthorized errors through Knox with no corresponding error in dlm.log.

Trusted-proxy prerequisites

hadoop.proxyuser.knox.hosts and hadoop.proxyuser.knox.groups must be present in the cluster configuration that the Cloudera Lakehouse Optimizer server loads, such as core-site.xml or dlm-cluster.properties. With dlm.security.trusted.proxy=true, Cloudera Lakehouse Optimizer builds its trusted-proxy allow list from hadoop.proxyuser.knox.hosts. If absent, the list is empty and every proxied request is rejected with the Rejecting doAs … from untrusted sender message.
important
The core-site.xml that Cloudera Lakehouse Optimizer loads is the same cluster-level file that Cloudera Manager provides. Cloudera Lakehouse Optimizer adds the core-site.xml file as a resource from the hadoop-confservice directory to its classpath. For example, configuration.addResource("core-site.xml") and configuration.addResource("hdfs-site.xml").
The end-user identity must be forwarded as the doAs query parameter (?doAs=<user>). Groups must be forwarded in the X-Knox-Actor-Groups-1 header. Cloudera Lakehouse Optimizer maps the group to the role through dlm.security.role.admin (default mapping: dlm_admin → admin). If the header is missing, Cloudera Lakehouse Optimizer logs doAs user <u>, groups null and returns 403 Forbidden.
dlm.security.authkey.path is optional. It applies only when Cloudera Lakehouse Optimizer must be authenticatd directly with JWTs instead of through an Apache Knox gateway. Set the property to a JWK (JSON) file that contains an RSA key pair, including the private key. If the property is not set or the file is missing or unreadable, Cloudera Lakehouse Optimizer generates a temporary in-memory key. Tokens are invalidated on restart and are not shared between HA nodes. If the file exists but cannot be parsed or contains only a public key, Cloudera Lakehouse Optimizer logs Failed to start security and continues running, but API calls are rejected (typically 403 Forbidden).

Knox gateway path and query/header contract

When you call REST APIs through Knox, use the Knox-proxied path (for example, https://<ENDPOINT>/cdp-proxy-token/clo/api/v1/config/health) with the Authorization: Bearer header. In addition, Knox must forward the end-user identity as the doAs query parameter and group membership in the X-Knox-Actor-Groups-1 header.

Without the trusted-proxy contract, the following call returns 403 Forbidden or 401 Unauthorized:

curl 'https://<ENDPOINT>/cdp-proxy-token/clo/api/v1/config/health' -H 'Authorization: Bearer <JWT>'

Working call with the prerequisites met:

curl "http://<clo-host>:8085/clo/api/v1/config/health?doAs=clodemo" -H "X-Knox-Actor-Groups-1: dlm_admin"

Troubleshooting REST API authentication


Condition	Cause
`403 Forbidden` with no entry in dlm.log	Security failed to initialize (authentication key)
`Rejecting doAs … from untrusted sender`	Missing `hadoop.proxyuser.knox.*` properties
`doAs user X, groups null`	Missing `X-Knox-Actor-Groups-1` header
`401 Unauthorized` with `Couldn't find bearer token`	Identity not forwarded as `?doAs=`