Manages pool of Spark sessions over HTTP
Livy, REST, Spark, http
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
| Display Name | API Name | Default Value | Allowable Values | Description |
|---|---|---|---|---|
| Livy Host | livy-cs-livy-host | The hostname (or IP address) of the Livy server. Supports Expression Language: true (will be evaluated using variable registry only) | ||
| Livy Port | livy-cs-livy-port | 8998 | The port number for the Livy server. Supports Expression Language: true (will be evaluated using variable registry only) | |
| Session Pool Size | livy-cs-session-pool-size | 2 | Number of sessions to keep open Supports Expression Language: true (will be evaluated using variable registry only) | |
| Session Type | livy-cs-session-kind | spark |
| The type of Spark session to start (spark, pyspark, pyspark3, sparkr, e.g.) |
| Session Manager Status Interval | livy-cs-session-manager-status-interval | 2 sec | The amount of time to wait between requesting session information updates. Supports Expression Language: true (will be evaluated using variable registry only) | |
| SSL Context Service | SSL Context Service | Controller Service API: SSLContextService Implementations: StandardRestrictedSSLContextService StandardSSLContextService | The SSL Context Service used to provide client certificate information for TLS/SSL (https) connections. | |
| Connection Timeout | Connection Timeout | 5 secs | Max wait time for connection to remote service. | |
| Session JARs | livy-cs-session-jars | JARs to be used in the Spark session. This property expects a comma-separated list of file resources. Supports Expression Language: true (will be evaluated using variable registry only) | ||
| Session Files | livy-cs-session-files | Files to be used in the Spark session. This property expects a comma-separated list of file resources. Supports Expression Language: true (will be evaluated using variable registry only) | ||
| Kerberos Credentials Service | kerberos-credentials-service | Controller Service API: KerberosCredentialsService Implementation: KeytabCredentialsService | Specifies the Kerberos Credentials Controller Service that should be used for authenticating with Kerberos |