Recipe and cluster template parameters
The following supported parameters can be specified as variables/dynamic parameters in recipes or cluster templates by using mustache formatting with "{{{ }}}" syntax.
The parameter keys listed below follow the following general conventions:
{ }indicates that the parameter key has multiple supported values, which are provided in this documentation. For example{fileSystemType}can be one of the following: s3, adls_gen_2, or wasb.[index]indicates that the parameter includes an index value for examplesharedService.datalakeComponents.[index]can be "sharedService.datalakeComponents.[0]", "sharedService.datalakeComponents.[1]", and so on. There is no easy way to find out what the index will be, but you may still be able to use these parameters (for example by creating a condition to filter them).
For information on how to set these parameters dynamically in a cluster template, refer to Setting custom properties.
Custom properties
Any custom property specified in the cluster template can be used as a recipe parameter. Refer to Custom properties documentation.
General
The general parameter group includes parameters related to general cluster
configuration.
| Description | Example key | Example value |
|---|---|---|
| Name of stack | general.stackName | teststack |
| UUID of cluster | general.uuid | 9aab7fdb-8940-454b-bc0a-62f04bce6519 |
| Cloudera Manager user name | general.cmUserName | |
| Cloudera Manager password | general.cmPassword | |
| Cloudera Manager IP | general.clusterManagerIp | 127.0.0.1 |
| Number of nodes | general.nodeCount | 5 |
| FQDN of primary gateway instance | general.primaryGatewayInstanceDiscoveryFQDN | ip-10-0-88-28.example.com |
| Number indicating the Kafka replication factor (3 or 1) | general.kafkaReplicationFactor | 1 |
Blueprint
The blueprint parameter group includes parameters related to cluster template
configuration.
| Parameter key | Description | Example key | Example value |
|---|---|---|---|
| blueprint.blueprintText | Blueprint text in JSON format | blueprint.blueprintText | |
| blueprint.version | Version of blueprint | blueprint.version | 7.2.8 |
Cloud storage
The fileSystemConfigs parameter group includes parameters related to cloud
storage configuration.
When forming the parameter keys, the {fileSystemType} should be replaced with
an actual cloud storage type such as "s3", "adls_gen_2", or "wasb".
| Parameter key | Description | Example key | Example value |
|---|---|---|---|
| File system common configurations | |||
| fileSystemConfigs.{fileSystemType}.storageContainer | Name of container in Azure storage account (Cloudbreak + stackId) | fileSystemConfigs.s3.storageContainer | cloudbreak123 |
| fileSystemConfigs.{fileSystemType}.type | Type of filesystem | fileSystemConfigs.s3.type | S3 |
| fileSystemConfigs.{fileSystemType}.locations.[index].configFile | Configuration file used to configure the filesystem | fileSystemConfigs.s3.locations.[0].configFile | hbase-site |
| fileSystemConfigs.{fileSystemType}.locations.[index].property | Property key of filesystem path in defined config | fileSystemConfigs.s3.locations.[0].property | hbase.rootdir |
| fileSystemConfigs.{fileSystemType}.locations.[index].value | Value of filesystem path in defined config | fileSystemConfigs.s3.locations.[0].value | s3a://testranger/testrecipe2/apps/hbase/data |
| Amazon S3 configurations | |||
| fileSystemConfigs.s3.storageContainer | Generated name (cloudbreak + stack id number) | fileSystemConfigs.s3.storageContainer | cloudbreak7941 |
| fileSystemConfigs.s3.locations.[index].configFile | Hadoop component configuration file | fileSystemConfigs.s3.locations.[0].configFile | zeppelin-site |
| fileSystemConfigs.s3.locations.[index].property | Component property name | fileSystemConfigs.s3.locations.[0].property | zeppelin.notebook.dir |
| fileSystemConfigs.s3.locations.[index].value | Component property value | fileSystemConfigs.s3.locations.[0].value | s3a://storagename/clustername/zeppelin/notebook |
| ADLS Gen2 configurations | |||
| fileSystemConfigs.adls_gen_2.accountName | Name of the corresponding Azure storage account | fileSystemConfigs.adls_gen_2.accountName | teststorageaccount |
| fileSystemConfigs.adls_gen_2.storageContainerName | Name of container in Azure storage account | fileSystemConfigs.adls_gen_2.storageContainerName | testcontainer |
External database
The rds parameter group includes parameters related to external database
configuration.
When forming the parameter keys, the {rdsType} should be replaced with the
actual database type such as "cloudera_manager", "beacon", "druid", "hive", "oozie", "ranger",
"superset", or some other user-defined type.
| Parameter key | Description | Example key | Example value |
|---|---|---|---|
| rds.{rdsType}.connectionURL | JDBC connection URL | rds.hive.connectionURL | Value is specified in the following format: jdbc:postgresql://host:port/database |
| rds.{rdsType}.connectionDriver | JDBC driver used for connection | rds.hive.connectionDriver | org.postgresql.Driver |
| rds.{rdsType}.connectionUserName | Username used for the JDBC connection | rds.hive.connectionUserName | testuser |
| rds.{rdsType}.connectionPassword | Password used for the JDBC connection | rds.hive.connectionPassword | TestPssword123 |
| rds.{rdsType}.databaseName | Target database of the JDBC connection | rds.hive.databaseName | myhivedb |
| rds.{rdsType}.host | Host of the JDBC connection | rds.hive.host | mydbhost |
| rds.{rdsType}.hostWithPortWithJdbc | Host of JDBC connection with port and JDBC prefix | rds.hive.hostWithPortWithJdbc | Value is specified in the following format: jdbc:postgresql://host:port |
| rds.{rdsType}.subprotocol | Sub-protocol from the JDBC URL | rds.hive.subprotocol | postgresql |
| rds.{rdsType}.connectionString | URL of JDBC the connection. In case of Ranger, this does not contain the port | rds.hive.connectionString | Value is specified in the following format: jdbc:postgresql://host:port/database |
| rds.{rdsType}.databaseVendor | Database vendor | rds.hive.databaseVendor | POSTGRES |
| rds.{rdsType}.withoutJDBCPrefix | URL of the JDBC connection without JDBC prefix | rds.hive.withoutJDBCPrefix | Value is specified in the following format: host:port/database |
Gateway
The gateway parameter group includes parameters related to Knox gateway
configuration.
| Parameter key | Description | Example key | Example value |
|---|---|---|---|
| gateway.ssoProvider | Path to the SSO provider | gateway.ssoProvider | /test/sso/api/v1/websso |
| gateway.signKey | Base64 encoded signing key | gateway.signKey | |
| gateway.signPub | Signing certificate (x509 format) | gateway.signPub | |
| gateway.signCert | Public SSH key used for signing (standard public key format) | gateway.signCert |
Shared services
The sharedService parameter group includes parameters related to Data Lake
configuration.
| Parameter key | Description | Example key | Example value |
|---|---|---|---|
| sharedService.rangerAdminPassword | Admin password of the Ranger component | sharedService.rangerAdminPassword | Admin1234! |
| sharedService.datalakeCluster | Flag indicating that the cluster is a data lake cluster | sharedService.datalakeCluster | true |
| sharedService.datalakeClusterManagerIp | Cloudera Manager IP of data lake cluster | sharedService.datalakeClusterManagerIp | 127.0.0.1 |
| sharedService.datalakeClusterManagerFqdn | Cloudera Manager FQDN of data lake cluster (or the IP if FQDN cannot be found) | sharedService.datalakeClusterManagerFqdn | ip-10-0-88-28.example.com |
