Hue is unable to create a Spark3 session with Livy3

Condition

This issue occurs when trying to open the Spark3 editor through Hue and receiving a pop-up displaying "Extra data: line 1 column 5."

Sample command to verify:

export LIVY_HOST=https://$(hsotname )-f:28998
curl -k --negotiate -u : -X POST --data '{"kind": "spark"}' -H "Content-Type: application/json" ${LIVY_HOST}/sessions

Example exception seen for the above command:

org.apache.hadoop.security.authorize.AuthorizationException: User:livy not allowed to do 'DECRYPT_EEK' on 'cdp_kms_key'

Cause

The issue arises due to insufficient permissions for the Livy user to perform theDECRYPT_EEK operation on the cdp_kms_key, which prevents the Livy session from initializing.

Solution

Grant the DECRYPT_EEK permission to the Livy user in Ranger. Once this permission is added, the Spark3 editor works correctly.