JEXL script

The Cloudera Lakehouse Optimizer Data Hub contains the default ClouderaAdaptive.jexl file that is a policy script. You can use the script as is. Otherwise, you can use the default JEXL file as a template to add or modify the contents, and then upload the file as a new JEXL file.

The following snippet shows a sample JEXL script:

// A sample jexl-based policy definition to evaluate the need for expire snapshot and generate action argument
#pragma dlm.statistics true
// Stores the list of generated actions
const actions = [...]
// Fetch iceberg stats
let stats = statistics($table);
if ($constants.expireSnapshot.enabled) {
    let min = $table['dlm.expireSnapshot.retainLast'] ?? $constants.expireSnapshot.retainLast ?? 2;
    let snapshots = stats.numberOfSnapshots;
    let snapshotTimestampDelta = stats.snapshotTimestampDelta ?? 3600000; 
    let deltaDurationThreshold = $table['dlm.expireSnapshot.snapshotsDurationDeltaMax'] ?? $constants.expireSnapshot.snapshotsDurationDeltaMax ?? 0;
    if (snapshots > min || snapshotTimestampDelta > deltaDurationThreshold) {
        // create an expire snapshot action
        actions.add(dlm:expireSnapshots($table, $constants));
    }
}
// return the list of actions
actions

The JEXL script contains the following variables, out of which, some variables are predefined:


Variables	Description
`$table`	Iceberg table object for which the policy is being evaluated. For example, `$table[‘dlmproperty’]` refers to the Iceberg table property named `dlmproperty` in the specified table.
`$constants`	Set of action argument values and defaults. For example, `$constants.expireSnapshot.retainLast` refers to the value for the `retainLast` argument for the `expireSnapshot` action.
`$catalog`	Iceberg catalog object.

The policy script supports retrieving the values from the table property or the policy constants. For example, let filesCountDrop =$table['dlm.rewriteDataFiles.filesCountDrop'] ?? $constants.rewriteDataFiles.filesCountDrop ?? 0.15;.

For more information about modifying the JEXL script contents, see JEXL syntax.