Cloudera Docs

Referencing Parameters

Parameter Reference Syntax

To configure an eligible property to reference a Parameter, use the # symbol as the start, with the Parameter's name enclosed in curly braces:

#{Parameter.Name}

This can be escaped using an additional # character at the beginning. To illustrate this, assume that the Parameter abc has a value of xxx and Parameter def has a value of yyy. Then, the following user-defined property values will evaluate to these effective values:

User-Entered Literal Property Value

Effective Property Value

Explanation

#{abc}

xxx

Simple substitution

#{abc}/data

xxx/data

Simple substitution with additional literal data

#{abc}/#{def}

xxx/yyy

Multiple substitution with additional literal data

#{abc

#{abc

No { } for parameter replacement

#abc

#abc

No { } for parameter replacement

##{abc}

#{abc}

Escaped # for literal interpretation

###{abc}

#xxx

Escaped # for literal interpretation, followed by simple substitution

####{abc}

##{abc}

Escaped # for literal interpretation, twice

#####{abc}

##xxx

Escaped # for literal interpretation, twice, followed by simple substitution

#{abc/data}

Exception thrown on property set operation

/ not a valid parameter name character

When referencing a Parameter, the Parameter reference is evaluated first. As an example, to replace xxx with zzz for the abc Parameter:

${ #{abc}:replace('xxx', 'zzz') }

Referencing and Creating Parameters During Component Configuration

Parameters can be easily referenced or created as you configure the components in your flow. For example, assume a process group has the Parameter Context "Kafka Settings" assigned to it. "Kafka Settings" contains the parameters kafka.broker and kafka.topic1.

To reference kafka.broker as the value for the "Kafka Brokers" property in the PublishKafka processor, clear the default value and begin a new entry with the start delimiter #{. Next use the keystroke control+space to show the list of available parameters:

Select kafka.broker and complete the entry with a closing curly brace }.

Help text describing this process is displayed when hovering over the Expression Language and Parameters eligibility indicators.

Parameters can also be created on the fly. For example, to create a parameter for the "Topic Name" property, select the "Convert to Parameter" icon (
) in that property's row. This icon will only be available if the user has appropriate permissions to modify the Parameter Context (see Accessing Parameters for more information).

The Add Parameter dialog will open. Configure the new parameter as desired.

Select "Apply". The process group's Parameter Context will be updated and the new parameter will be referenced by the property with the proper syntax applied automatically.

Properties values that are selectable can also reference parameters. In addition to applying the "Convert to Parameter" method described earlier, the option "Reference parameter.." is available in the value drop-down menu.

Selecting "Reference parameter…​" will display a drop-down list of available parameters, determined by the parameter context assigned to the component's process group and the user's access policies.

Hovering over the question mark icon (
) displays the parameter's description.

Using Parameters with Sensitive Properties

Sensitive properties may only reference sensitive Parameters. This is important for Versioning a DataFlow. The value of the sensitive Parameter itself will NOT be sent to the flow registry, only the fact that the property references the sensitive Parameter. For more information see Parameters in Versioned Flows.

The value of a sensitive property must be set to a single Parameter reference. For example, values of #{password}123 and #{password}#{suffix} are not allowed. Sending #{password}123 would lead to exposing part of the sensitive property's value. This is in contrast to a non-sensitive property, where a value such as #{path}/child/file.txt is valid.