Flink Properties in Cloudera Runtime 7.2.16
Role groups:
Flink Dashboard
Advanced
Flink Dashboard Advanced Configuration Snippet (Safety Valve) for flink-conf/flink-conf.yaml
- Description
- For advanced use only. A string to be inserted into flink-conf/flink-conf.yaml for this role only.
- Related Name
- Default Value
- API Name
-
flink-conf/flink-conf.yaml_role_safety_valve
- Required
- false
Flink Dashboard Environment Advanced Configuration Snippet (Safety Valve)
- Description
- For advanced use only, key-value pairs (one on each line) to be inserted into a role's environment. Applies to configurations of this role except client configuration.
- Related Name
- Default Value
- API Name
-
FLINK_HISTORY_SERVER_role_env_safety_valve
- Required
- false
Flink Dashboard Logging Advanced Configuration Snippet (Safety Valve)
- Description
- For advanced use only, a string to be inserted into log4j.properties for this role only.
- Related Name
- Default Value
- API Name
-
log4j_safety_valve
- Required
- false
Enable auto refresh for metric configurations
- Description
- When true, Enable Metric Collection and Metric Filter parameters will be set automatically if they're changed. Otherwise, a refresh by hand is required.
- Related Name
- Default Value
- false
- API Name
-
metric_config_auto_refresh
- Required
- false
Heap Dump Directory
- Description
- Path to directory where heap dumps are generated when java.lang.OutOfMemoryError error is thrown. This directory is automatically created if it does not exist. If this directory already exists, it will be owned by the current role user with 1777 permissions. Sharing the same directory among multiple roles will cause an ownership race. The heap dump files are created with 600 permissions and are owned by the role user. The amount of free space in this directory should be greater than the maximum Java Process heap size configured for this role.
- Related Name
-
oom_heap_dump_dir
- Default Value
- /tmp
- API Name
-
oom_heap_dump_dir
- Required
- false
Dump Heap When Out of Memory
- Description
- When set, generates a heap dump file when when an out-of-memory error occurs.
- Related Name
- Default Value
- true
- API Name
-
oom_heap_dump_enabled
- Required
- true
Kill When Out of Memory
- Description
- When set, a SIGKILL signal is sent to the role process when java.lang.OutOfMemoryError is thrown.
- Related Name
- Default Value
- true
- API Name
-
oom_sigkill_enabled
- Required
- true
Automatically Restart Process
- Description
- When set, this role's process is automatically (and transparently) restarted in the event of an unexpected failure. This configuration applies in the time after the Start Wait Timeout period.
- Related Name
- Default Value
- false
- API Name
-
process_auto_restart
- Required
- true
Enable Metric Collection
- Description
- Cloudera Manager agent monitors each service and each of its role by publishing metrics to the Cloudera Manager Service Monitor. Setting it to false will stop Cloudera Manager agent from publishing any metric for corresponding service/roles. This is usually helpful for services that generate large amount of metrics which Service Monitor is not able to process.
- Related Name
- Default Value
- true
- API Name
-
process_should_monitor
- Required
- true
Process Start Retry Attempts
- Description
- Number of times to try starting a role's process when the process exits before the Start Wait Timeout period. After a process is running beyond the Start Wait Timeout, the retry count is reset. Setting this configuration to zero will prevent restart of the process during the Start Wait Timeout period.
- Related Name
- Default Value
- 3
- API Name
-
process_start_retries
- Required
- false
Process Start Wait Timeout
- Description
- The time in seconds to wait for a role's process to start successfully on a host. Processes which exit/crash before this time will be restarted until reaching the limit specified by the Start Retry Attempts count parameter. Setting this configuration to zero will turn off this feature.
- Related Name
- Default Value
- 20
- API Name
-
process_start_secs
- Required
- false
Logs
Flink Dashboard Log Directory
- Description
- The log directory for log files of the role Flink Dashboard.
- Related Name
-
log.dir
- Default Value
- /var/log/flink
- API Name
-
log_dir
- Required
- false
Flink Dashboard Logging Threshold
- Description
- The minimum log level for Flink Dashboard logs
- Related Name
- Default Value
- INFO
- API Name
-
log_threshold
- Required
- false
Flink Dashboard Maximum Log File Backups
- Description
- The maximum number of rolled log files to keep for Flink Dashboard logs. Typically used by log4j or logback.
- Related Name
- Default Value
- 10
- API Name
-
max_log_backup_index
- Required
- false
Flink Dashboard Max Log Size
- Description
- The maximum size, in megabytes, per log file for Flink Dashboard logs. Typically used by log4j or logback.
- Related Name
- Default Value
- 200 MiB
- API Name
-
max_log_size
- Required
- false
Monitoring
Enable Health Alerts for this Role
- Description
- When set, Cloudera Manager will send alerts when the health of this role reaches the threshold specified by the EventServer setting eventserver_health_events_alert_threshold
- Related Name
- Default Value
- true
- API Name
-
enable_alerts
- Required
- false
Enable Configuration Change Alerts
- Description
- When set, Cloudera Manager will send alerts when this entity's configuration changes.
- Related Name
- Default Value
- false
- API Name
-
enable_config_alerts
- Required
- false
File Descriptor Monitoring Thresholds
- Description
- The health test thresholds of the number of file descriptors used. Specified as a percentage of file descriptor limit.
- Related Name
- Default Value
- Warning: 50.0 %, Critical: 70.0 %
- API Name
-
flink_history_server_fd_thresholds
- Required
- false
Flink Dashboard Host Health Test
- Description
- When computing the overall Flink Dashboard health, consider the host's health.
- Related Name
- Default Value
- true
- API Name
-
flink_history_server_host_health_enabled
- Required
- false
Flink Dashboard Process Health Test
- Description
- Enables the health test that the Flink Dashboard's process state is consistent with the role configuration
- Related Name
- Default Value
- true
- API Name
-
flink_history_server_scm_health_enabled
- Required
- false
Log Directory Free Space Monitoring Absolute Thresholds
- Description
- The health test thresholds for monitoring of free space on the filesystem that contains this role's log directory.
- Related Name
- Default Value
- Warning: 10 GiB, Critical: 5 GiB
- API Name
-
log_directory_free_space_absolute_thresholds
- Required
- false
Log Directory Free Space Monitoring Percentage Thresholds
- Description
- The health test thresholds for monitoring of free space on the filesystem that contains this role's log directory. Specified as a percentage of the capacity on that filesystem. This setting is not used if a Log Directory Free Space Monitoring Absolute Thresholds setting is configured.
- Related Name
- Default Value
- Warning: Never, Critical: Never
- API Name
-
log_directory_free_space_percentage_thresholds
- Required
- false
Metric Filter
- Description
- Defines a Metric Filter for this role. Cloudera Manager Agents will not send filtered metrics to the Service Monitor. Define the following fields:
- Health Test Metric Set - Select this parameter to collect only metrics required for health tests.
- Default Dashboard Metric Set - Select this parameter to collect only metrics required for the default dashboards. For user-defined charts, you must add the metrics you require for the chart using the Custom Metrics parameter.
- Include/Exclude Custom Metrics - Select Include to specify metrics that should be collected. Select Exclude to specify metrics that should not be collected. Enter the metric names to be included or excluded using the Metric Name parameter.
- Metric Name - The name of a metric that will be included or excluded during metric collection.
jvm_heap_used_mb
metric:- Include only Health Test Metric Set: Selected.
- Include/Exclude Custom Metrics: Set to Include.
- Metric Name:
jvm_heap_used_mb
{ "includeHealthTestMetricSet": true, "filterType": "whitelist", "metrics": ["jvm_heap_used_mb"] }
- Related Name
- Default Value
- API Name
-
monitoring_metric_filter
- Required
- false
Swap Memory Usage Rate Thresholds
- Description
- The health test thresholds on the swap memory usage rate of the process. Specified as the change of the used swap memory during the predefined period.
- Related Name
- Default Value
- Warning: Never, Critical: Never
- API Name
-
process_swap_memory_rate_thresholds
- Required
- false
Swap Memory Usage Rate Window
- Description
- The period to review when computing unexpected swap memory usage change of the process.
- Related Name
-
common.process.swap_memory_rate_window
- Default Value
- 5 minute(s)
- API Name
-
process_swap_memory_rate_window
- Required
- false
Process Swap Memory Thresholds
- Description
- The health test thresholds on the swap memory usage of the process. This takes precedence over the host level threshold.
- Related Name
- Default Value
- Warning: 200 B, Critical: Never
- API Name
-
process_swap_memory_thresholds
- Required
- false
Role Triggers
- Description
- The configured triggers for this role. This is a JSON-formatted list of triggers. These triggers are evaluated as part as the health system. Every trigger expression is parsed, and if the trigger condition is met, the list of actions provided in the trigger expression is executed. Each trigger has the following fields:
triggerName
(mandatory) - The name of the trigger. This value must be unique for the specific role.triggerExpression
(mandatory) - A tsquery expression representing the trigger.streamThreshold
(optional) - The maximum number of streams that can satisfy a condition of a trigger before the condition fires. By default set to 0, and any stream returned causes the condition to fire.enabled
(optional) - By default set to 'true'. If set to 'false', the trigger is not evaluated.expressionEditorConfig
(optional) - Metadata for the trigger editor. If present, the trigger should only be edited from the Edit Trigger page; editing the trigger here can lead to inconsistencies.
[{"triggerName": "sample-trigger", "triggerExpression": "IF (SELECT fd_open WHERE roleName=$ROLENAME and last(fd_open) > 1500) DO health:bad", "streamThreshold": 0, "enabled": "true"}]
See the trigger rules documentation for more details on how to write triggers using tsquery.The JSON format is evolving and may change and, as a result, backward compatibility is not guaranteed between releases. - Related Name
- Default Value
- []
- API Name
-
role_triggers
- Required
- true
Unexpected Exits Thresholds
- Description
- The health test thresholds for unexpected exits encountered within a recent period specified by the unexpected_exits_window configuration for the role.
- Related Name
- Default Value
- Warning: Never, Critical: Any
- API Name
-
unexpected_exits_thresholds
- Required
- false
Unexpected Exits Monitoring Period
- Description
- The period to review when computing unexpected exits.
- Related Name
- Default Value
- 5 minute(s)
- API Name
-
unexpected_exits_window
- Required
- false
Other
History Server Archive Directory (HDFS)
- Description
- Comma separated list of directories to monitor for completed jobs.
- Related Name
-
historyserver.archive.fs.dir
- Default Value
- /user/flink/applicationHistory
- API Name
-
historyserver_archive_fs_dir
- Required
- true
History Server Archive Directory Refresh Interval
- Description
- Interval in milliseconds for refreshing the archived job directories.
- Related Name
-
historyserver.archive.fs.refresh-interval
- Default Value
- 10000
- API Name
-
historyserver_archive_fs_refresh_interval
- Required
- true
History Server Archive Retained Jobs
- Description
- The maximum number of jobs to retain in each archive directory.
- Related Name
-
historyserver.archive.retained-jobs
- Default Value
- 50
- API Name
-
historyserver_archive_retained_jobs
- Required
- true
Flink Dashboard Cluster Fetcher
- Description
- Flink Dashboard Cluster Fetcher
- Related Name
-
historyserver.cluster.fetcher
- Default Value
- YARN
- API Name
-
historyserver_cluster_fetcher
- Required
- true
Use SPNEGO Authentication
- Description
- Enables SPNEGO authentication.
- Related Name
-
historyserver.security.spnego.auth.enabled
- Default Value
- false
- API Name
-
historyserver_security_spnego_auth_enabled
- Required
- true
Kerberos Login Keytab
- Description
- Absolute path to a Kerberos keytab file that contains the user credentials.
- Related Name
-
security.kerberos.login.keytab
- Default Value
- flink.keytab
- API Name
-
security_kerberos_login_keytab
- Required
- false
Performance
Maximum Process File Descriptors
- Description
- If configured, overrides the process soft and hard rlimits (also called ulimits) for file descriptors to the configured value.
- Related Name
- Default Value
- API Name
-
rlimit_fds
- Required
- false
Ports and Addresses
Flink Dashboard Port
- Description
- Port of the Flink Dashboard's web interface.
- Related Name
-
historyserver.web.port
- Default Value
- 18211
- API Name
-
historyserver_web_port
- Required
- true
Resource Management
Cgroup CPU Shares
- Description
- Number of CPU shares to assign to this role. The greater the number of shares, the larger the share of the host's CPUs that will be given to this role when the host experiences CPU contention. Must be between 2 and 262144. Defaults to 1024 for processes not managed by Cloudera Manager.
- Related Name
-
cpu.shares
- Default Value
- 1024
- API Name
-
rm_cpu_shares
- Required
- true
Custom Control Group Resources (overrides Cgroup settings)
- Description
- Custom control group resources to assign to this role, which will be enforced by the Linux kernel. These resources should exist on the target hosts, otherwise an error will occur when the process starts. Use the same format as used for arguments to the cgexec command: resource1,resource2:path1 or resource3:path2 For example: 'cpu,memory:my/path blkio:my2/path2' ***These settings override other cgroup settings.***
- Related Name
-
custom.cgroups
- Default Value
- API Name
-
rm_custom_resources
- Required
- false
Cgroup I/O Weight
- Description
- Weight for the read I/O requests issued by this role. The greater the weight, the higher the priority of the requests when the host experiences I/O contention. Must be between 100 and 1000. Defaults to 1000 for processes not managed by Cloudera Manager.
- Related Name
-
blkio.weight
- Default Value
- 500
- API Name
-
rm_io_weight
- Required
- true
Cgroup Memory Hard Limit
- Description
- Hard memory limit to assign to this role, enforced by the Linux kernel. When the limit is reached, the kernel will reclaim pages charged to the process. If reclaiming fails, the kernel may kill the process. Both anonymous as well as page cache pages contribute to the limit. Use a value of -1 to specify no limit. By default processes not managed by Cloudera Manager will have no limit. If the value is -1, Cloudera Manager will not monitor Cgroup memory usage therefore some of the charts will show 'No Data'
- Related Name
-
memory.limit_in_bytes
- Default Value
- -1 MiB
- API Name
-
rm_memory_hard_limit
- Required
- true
Cgroup Memory Soft Limit
- Description
- Soft memory limit to assign to this role, enforced by the Linux kernel. When the limit is reached, the kernel will reclaim pages charged to the process if and only if the host is facing memory pressure. If reclaiming fails, the kernel may kill the process. Both anonymous as well as page cache pages contribute to the limit. Use a value of -1 to specify no limit. By default processes not managed by Cloudera Manager will have no limit. If the value is -1, Cloudera Manager will not monitor Cgroup memory usage therefore some of the charts will show 'No Data'
- Related Name
-
memory.soft_limit_in_bytes
- Default Value
- -1 MiB
- API Name
-
rm_memory_soft_limit
- Required
- true
Security
Flink Dashboard TLS/SSL Trust Store File
- Description
- The location on disk of the trust store, in .jks format, used to confirm the authenticity of TLS/SSL servers that Flink Dashboard might connect to. This trust store must contain the certificate(s) used to sign the service(s) connected to. If this parameter is not provided, the default list of well-known certificate authorities is used instead.
- Related Name
-
security.ssl.rest.truststore
- Default Value
- API Name
-
ssl_client_truststore_location
- Required
- false
Flink Dashboard TLS/SSL Trust Store Password
- Description
- The password for the Flink Dashboard TLS/SSL Trust Store File. This password is not required to access the trust store; this field can be left blank. This password provides optional integrity checking of the file. The contents of trust stores are certificates, and certificates are public information.
- Related Name
-
security.ssl.rest.truststore-password
- Default Value
- API Name
-
ssl_client_truststore_password
- Required
- false
Enable TLS/SSL for Flink Dashboard
- Description
- Encrypt communication between clients and Flink Dashboard using Transport Layer Security (TLS) (formerly known as Secure Socket Layer (SSL)).
- Related Name
-
ssl_enabled
- Default Value
- false
- API Name
-
ssl_enabled
- Required
- false
Flink Dashboard TLS/SSL Server Keystore Key Password
- Description
- The password that protects the private key contained in the keystore used when Flink Dashboard is acting as a TLS/SSL server.
- Related Name
-
security.ssl.rest.key-password
- Default Value
- API Name
-
ssl_server_keystore_keypassword
- Required
- false
Flink Dashboard TLS/SSL Server Keystore File Location
- Description
- The path to the TLS/SSL keystore file containing the server certificate and private key used for TLS/SSL. Used when Flink Dashboard is acting as a TLS/SSL server. The keystore must be in the format specified in Administration > Settings > Java Keystore Type.
- Related Name
-
security.ssl.rest.keystore
- Default Value
- API Name
-
ssl_server_keystore_location
- Required
- false
Flink Dashboard TLS/SSL Server Keystore File Password
- Description
- The password for the Flink Dashboard keystore file.
- Related Name
-
security.ssl.rest.keystore-password
- Default Value
- API Name
-
ssl_server_keystore_password
- Required
- false
Stacks Collection
Stacks Collection Data Retention
- Description
- The amount of stacks data that is retained. After the retention limit is reached, the oldest data is deleted.
- Related Name
-
stacks_collection_data_retention
- Default Value
- 100 MiB
- API Name
-
stacks_collection_data_retention
- Required
- false
Stacks Collection Directory
- Description
- The directory in which stacks logs are placed. If not set, stacks are logged into a
stacks
subdirectory of the role's log directory. If this directory already exists, it will be owned by the current role user with 755 permissions. Sharing the same directory among multiple roles will cause an ownership race. - Related Name
-
stacks_collection_directory
- Default Value
- API Name
-
stacks_collection_directory
- Required
- false
Stacks Collection Enabled
- Description
- Whether or not periodic stacks collection is enabled.
- Related Name
-
stacks_collection_enabled
- Default Value
- false
- API Name
-
stacks_collection_enabled
- Required
- true
Stacks Collection Frequency
- Description
- The frequency with which stacks are collected.
- Related Name
-
stacks_collection_frequency
- Default Value
- 5.0 second(s)
- API Name
-
stacks_collection_frequency
- Required
- false
Stacks Collection Method
- Description
- The method used to collect stacks. The jstack option involves periodically running the jstack command against the role's daemon process. The servlet method is available for those roles that have an HTTP server endpoint exposing the current stacks traces of all threads. When the servlet method is selected, that HTTP endpoint is periodically scraped.
- Related Name
-
stacks_collection_method
- Default Value
- jstack
- API Name
-
stacks_collection_method
- Required
- false
Suppressions
Suppress Configuration Validator: CDH Version Validator
- Description
- Whether to suppress configuration warnings produced by the CDH Version Validator configuration validator.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_cdh_version_validator
- Required
- true
Suppress Parameter Validation: Flink Dashboard Advanced Configuration Snippet (Safety Valve) for flink-conf/flink-conf.yaml
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Flink Dashboard Advanced Configuration Snippet (Safety Valve) for flink-conf/flink-conf.yaml parameter.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_flink-conf/flink-conf.yaml_role_safety_valve
- Required
- true
Suppress Parameter Validation: Flink Dashboard Environment Advanced Configuration Snippet (Safety Valve)
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Flink Dashboard Environment Advanced Configuration Snippet (Safety Valve) parameter.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_flink_history_server_role_env_safety_valve
- Required
- true
Suppress Parameter Validation: History Server Archive Directory (HDFS)
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the History Server Archive Directory (HDFS) parameter.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_historyserver_archive_fs_dir
- Required
- true
Suppress Parameter Validation: Flink Dashboard Port
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Flink Dashboard Port parameter.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_historyserver_web_port
- Required
- true
Suppress Parameter Validation: Flink Dashboard Logging Advanced Configuration Snippet (Safety Valve)
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Flink Dashboard Logging Advanced Configuration Snippet (Safety Valve) parameter.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_log4j_safety_valve
- Required
- true
Suppress Parameter Validation: Flink Dashboard Log Directory
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Flink Dashboard Log Directory parameter.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_log_dir
- Required
- true
Suppress Parameter Validation: Heap Dump Directory
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Heap Dump Directory parameter.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_oom_heap_dump_dir
- Required
- true
Suppress Parameter Validation: Custom Control Group Resources (overrides Cgroup settings)
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Custom Control Group Resources (overrides Cgroup settings) parameter.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_rm_custom_resources
- Required
- true
Suppress Parameter Validation: Role Triggers
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Role Triggers parameter.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_role_triggers
- Required
- true
Suppress Parameter Validation: Kerberos Login Keytab
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Kerberos Login Keytab parameter.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_security_kerberos_login_keytab
- Required
- true
Suppress Parameter Validation: Flink Dashboard TLS/SSL Trust Store File
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Flink Dashboard TLS/SSL Trust Store File parameter.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_ssl_client_truststore_location
- Required
- true
Suppress Parameter Validation: Flink Dashboard TLS/SSL Trust Store Password
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Flink Dashboard TLS/SSL Trust Store Password parameter.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_ssl_client_truststore_password
- Required
- true
Suppress Parameter Validation: Flink Dashboard TLS/SSL Server Keystore Key Password
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Flink Dashboard TLS/SSL Server Keystore Key Password parameter.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_ssl_server_keystore_keypassword
- Required
- true
Suppress Parameter Validation: Flink Dashboard TLS/SSL Server Keystore File Location
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Flink Dashboard TLS/SSL Server Keystore File Location parameter.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_ssl_server_keystore_location
- Required
- true
Suppress Parameter Validation: Flink Dashboard TLS/SSL Server Keystore File Password
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Flink Dashboard TLS/SSL Server Keystore File Password parameter.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_ssl_server_keystore_password
- Required
- true
Suppress Parameter Validation: Stacks Collection Directory
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Stacks Collection Directory parameter.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_stacks_collection_directory
- Required
- true
Suppress Health Test: Audit Pipeline Test
- Description
- Whether to suppress the results of the Audit Pipeline Test heath test. The results of suppressed health tests are ignored when computing the overall health of the associated host, role or service, so suppressed health tests will not generate alerts.
- Related Name
- Default Value
- false
- API Name
-
role_health_suppression_flink_flink_history_server_audit_health
- Required
- true
Suppress Health Test: File Descriptors
- Description
- Whether to suppress the results of the File Descriptors heath test. The results of suppressed health tests are ignored when computing the overall health of the associated host, role or service, so suppressed health tests will not generate alerts.
- Related Name
- Default Value
- false
- API Name
-
role_health_suppression_flink_flink_history_server_file_descriptor
- Required
- true
Suppress Health Test: Host Health
- Description
- Whether to suppress the results of the Host Health heath test. The results of suppressed health tests are ignored when computing the overall health of the associated host, role or service, so suppressed health tests will not generate alerts.
- Related Name
- Default Value
- false
- API Name
-
role_health_suppression_flink_flink_history_server_host_health
- Required
- true
Suppress Health Test: Log Directory Free Space
- Description
- Whether to suppress the results of the Log Directory Free Space heath test. The results of suppressed health tests are ignored when computing the overall health of the associated host, role or service, so suppressed health tests will not generate alerts.
- Related Name
- Default Value
- false
- API Name
-
role_health_suppression_flink_flink_history_server_log_directory_free_space
- Required
- true
Suppress Health Test: Process Status
- Description
- Whether to suppress the results of the Process Status heath test. The results of suppressed health tests are ignored when computing the overall health of the associated host, role or service, so suppressed health tests will not generate alerts.
- Related Name
- Default Value
- false
- API Name
-
role_health_suppression_flink_flink_history_server_scm_health
- Required
- true
Suppress Health Test: Swap Memory Usage
- Description
- Whether to suppress the results of the Swap Memory Usage heath test. The results of suppressed health tests are ignored when computing the overall health of the associated host, role or service, so suppressed health tests will not generate alerts.
- Related Name
- Default Value
- false
- API Name
-
role_health_suppression_flink_flink_history_server_swap_memory_usage
- Required
- true
Suppress Health Test: Swap Memory Usage Rate Beta
- Description
- Whether to suppress the results of the Swap Memory Usage Rate Beta heath test. The results of suppressed health tests are ignored when computing the overall health of the associated host, role or service, so suppressed health tests will not generate alerts.
- Related Name
- Default Value
- false
- API Name
-
role_health_suppression_flink_flink_history_server_swap_memory_usage_rate
- Required
- true
Suppress Health Test: Unexpected Exits
- Description
- Whether to suppress the results of the Unexpected Exits heath test. The results of suppressed health tests are ignored when computing the overall health of the associated host, role or service, so suppressed health tests will not generate alerts.
- Related Name
- Default Value
- false
- API Name
-
role_health_suppression_flink_flink_history_server_unexpected_exits
- Required
- true
Gateway
Advanced
Deploy Directory
- Description
- The directory where the client configs will be deployed
- Related Name
- Default Value
- /etc/flink
- API Name
-
client_config_root_dir
- Required
- true
Flink Client Advanced Configuration Snippet (Safety Valve) for flink-conf/atlas-application.properties
- Description
- For advanced use only, a string to be inserted into the client configuration for flink-conf/atlas-application.properties.
- Related Name
- Default Value
- API Name
-
flink-conf/atlas-application.properties_client_config_safety_valve
- Required
- false
Flink Client Advanced Configuration Snippet (Safety Valve) for flink-conf/flink-conf.yaml
- Description
- For advanced use only, a string to be inserted into the client configuration for flink-conf/flink-conf.yaml.
- Related Name
- Default Value
- API Name
-
flink-conf/flink-conf.yaml_client_config_safety_valve
- Required
- false
Flink Client Advanced Configuration Snippet (Safety Valve) for flink-conf/log4j-cli.properties
- Description
- For advanced use only, a string to be inserted into the client configuration for flink-conf/log4j-cli.properties.
- Related Name
- Default Value
- API Name
-
flink-conf/log4j-cli.properties_client_config_safety_valve
- Required
- false
Flink Client Advanced Configuration Snippet (Safety Valve) for flink-conf/log4j.properties
- Description
- For advanced use only, a string to be inserted into the client configuration for flink-conf/log4j.properties.
- Related Name
- Default Value
- API Name
-
flink-conf/log4j.properties_client_config_safety_valve
- Required
- false
Flink Client Advanced Configuration Snippet (Safety Valve) for flink-conf/sql-client-defaults.yaml
- Description
- For advanced use only, a string to be inserted into the client configuration for flink-conf/sql-client-defaults.yaml.
- Related Name
- Default Value
- API Name
-
flink-conf/sql-client-defaults.yaml_client_config_safety_valve
- Required
- false
Monitoring
Enable Configuration Change Alerts
- Description
- When set, Cloudera Manager will send alerts when this entity's configuration changes.
- Related Name
- Default Value
- false
- API Name
-
enable_config_alerts
- Required
- false
Other
Alternatives Priority
- Description
- The priority level that the client configuration will have in the Alternatives system on the hosts. Higher priority levels will cause Alternatives to prefer this configuration over any others.
- Related Name
- Default Value
- 50
- API Name
-
client_config_priority
- Required
- true
Security
Gateway TLS/SSL Trust Store File
- Description
- The location on disk of the trust store, in .jks format, used to confirm the authenticity of TLS/SSL servers that Gateway might connect to. This trust store must contain the certificate(s) used to sign the service(s) connected to. If this parameter is not provided, the default list of well-known certificate authorities is used instead.
- Related Name
-
security.ssl.rest.truststore
- Default Value
- API Name
-
ssl_client_truststore_location
- Required
- false
Gateway TLS/SSL Trust Store Password
- Description
- The password for the Gateway TLS/SSL Trust Store File. This password is not required to access the trust store; this field can be left blank. This password provides optional integrity checking of the file. The contents of trust stores are certificates, and certificates are public information.
- Related Name
-
security.ssl.rest.truststore-password
- Default Value
- API Name
-
ssl_client_truststore_password
- Required
- false
Suppressions
Suppress Configuration Validator: CDH Version Validator
- Description
- Whether to suppress configuration warnings produced by the CDH Version Validator configuration validator.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_cdh_version_validator
- Required
- true
Suppress Parameter Validation: Deploy Directory
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Deploy Directory parameter.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_client_config_root_dir
- Required
- true
Suppress Parameter Validation: Flink Client Advanced Configuration Snippet (Safety Valve) for flink-conf/atlas-application.properties
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Flink Client Advanced Configuration Snippet (Safety Valve) for flink-conf/atlas-application.properties parameter.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_flink-conf/atlas-application.properties_client_config_safety_valve
- Required
- true
Suppress Parameter Validation: Flink Client Advanced Configuration Snippet (Safety Valve) for flink-conf/flink-conf.yaml
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Flink Client Advanced Configuration Snippet (Safety Valve) for flink-conf/flink-conf.yaml parameter.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_flink-conf/flink-conf.yaml_client_config_safety_valve
- Required
- true
Suppress Parameter Validation: Flink Client Advanced Configuration Snippet (Safety Valve) for flink-conf/log4j-cli.properties
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Flink Client Advanced Configuration Snippet (Safety Valve) for flink-conf/log4j-cli.properties parameter.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_flink-conf/log4j-cli.properties_client_config_safety_valve
- Required
- true
Suppress Parameter Validation: Flink Client Advanced Configuration Snippet (Safety Valve) for flink-conf/log4j.properties
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Flink Client Advanced Configuration Snippet (Safety Valve) for flink-conf/log4j.properties parameter.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_flink-conf/log4j.properties_client_config_safety_valve
- Required
- true
Suppress Parameter Validation: Flink Client Advanced Configuration Snippet (Safety Valve) for flink-conf/sql-client-defaults.yaml
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Flink Client Advanced Configuration Snippet (Safety Valve) for flink-conf/sql-client-defaults.yaml parameter.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_flink-conf/sql-client-defaults.yaml_client_config_safety_valve
- Required
- true
Suppress Parameter Validation: Gateway TLS/SSL Trust Store File
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Gateway TLS/SSL Trust Store File parameter.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_ssl_client_truststore_location
- Required
- true
Suppress Parameter Validation: Gateway TLS/SSL Trust Store Password
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Gateway TLS/SSL Trust Store Password parameter.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_ssl_client_truststore_password
- Required
- true
Service-Wide
Advanced
Flink Service Advanced Configuration Snippet (Safety Valve) for flink-conf/flink-conf.yaml
- Description
- For advanced use only, a string to be inserted into flink-conf/flink-conf.yaml. Applies to configurations of all roles in this service except client configuration.
- Related Name
- Default Value
- API Name
-
flink-conf/flink-conf.yaml_service_safety_valve
- Required
- false
Flink Service Environment Advanced Configuration Snippet (Safety Valve)
- Description
- For advanced use only, key-value pairs (one on each line) to be inserted into a role's environment. Applies to configurations of all roles in this service except client configuration.
- Related Name
- Default Value
- API Name
-
FLINK_service_env_safety_valve
- Required
- false
System Group
- Description
- The group that this service's processes should run as.
- Related Name
- Default Value
- flink
- API Name
-
process_groupname
- Required
- true
System User
- Description
- The user that this service's processes should run as.
- Related Name
- Default Value
- flink
- API Name
-
process_username
- Required
- true
Monitoring
Enable Service Level Health Alerts
- Description
- When set, Cloudera Manager will send alerts when the health of this service reaches the threshold specified by the EventServer setting eventserver_health_events_alert_threshold
- Related Name
- Default Value
- true
- API Name
-
enable_alerts
- Required
- false
Enable Configuration Change Alerts
- Description
- When set, Cloudera Manager will send alerts when this entity's configuration changes.
- Related Name
- Default Value
- false
- API Name
-
enable_config_alerts
- Required
- false
Service Triggers
- Description
- The configured triggers for this service. This is a JSON-formatted list of triggers. These triggers are evaluated as part as the health system. Every trigger expression is parsed, and if the trigger condition is met, the list of actions provided in the trigger expression is executed. Each trigger has the following fields:
triggerName
(mandatory) - The name of the trigger. This value must be unique for the specific service.triggerExpression
(mandatory) - A tsquery expression representing the trigger.streamThreshold
(optional) - The maximum number of streams that can satisfy a condition of a trigger before the condition fires. By default set to 0, and any stream returned causes the condition to fire.enabled
(optional) - By default set to 'true'. If set to 'false', the trigger is not evaluated.expressionEditorConfig
(optional) - Metadata for the trigger editor. If present, the trigger should only be edited from the Edit Trigger page; editing the trigger here can lead to inconsistencies.
[{"triggerName": "sample-trigger", "triggerExpression": "IF (SELECT fd_open WHERE roleType = DataNode and last(fd_open) > 500) DO health:bad", "streamThreshold": 10, "enabled": "true"}]
See the trigger rules documentation for more details on how to write triggers using tsquery.The JSON format is evolving and may change and, as a result, backward compatibility is not guaranteed between releases. - Related Name
- Default Value
- []
- API Name
-
service_triggers
- Required
- true
Service Monitor Derived Configs Advanced Configuration Snippet (Safety Valve)
- Description
- For advanced use only, a list of derived configuration properties that will be used by the Service Monitor instead of the default ones.
- Related Name
- Default Value
- API Name
-
smon_derived_configs_safety_valve
- Required
- false
Other
Enable Atlas Metadata Collection
- Description
- When enabling this make sure that your Flink gateway nodes also have the Atlas gateway role assigned.
- Related Name
-
atlas.collection.enabled
- Default Value
- false
- API Name
-
atlas_collection_enabled
- Required
- false
Atlas Metadata Namespace
- Description
- Metadata Namespace used in Atlas for Flink applications.
- Related Name
-
atlas.metadata.namespace
- Default Value
- cm
- API Name
-
atlas_metadata_namespace
- Required
- true
ATLAS Service
- Description
- Name of the ATLAS service that this Flink service instance depends on
- Related Name
- Default Value
- API Name
-
atlas_service
- Required
- false
Checkpointing Interval (milliseconds)
- Description
- Gets the interval in which checkpoints are periodically scheduled. This setting defines the base interval. Checkpoint triggering may be delayed by the settings execution.checkpointing.max-concurrent-checkpoints and execution.checkpointing.min-pause
- Related Name
-
execution.checkpointing.interval
- Default Value
- API Name
-
checkpointing_interval
- Required
- false
Min Pause Between Checkpoints (milliseconds)
- Description
- The minimal pause between checkpointing attempts. This setting defines how soon thecheckpoint coordinator may trigger another checkpoint after it becomes possible to triggeranother checkpoint with respect to the maximum number of concurrent checkpoints(see execution.checkpointing.max-concurrent-checkpoints).
- Related Name
-
execution.checkpointing.min-pause
- Default Value
- 0
- API Name
-
checkpointing_min_pause
- Required
- true
Checkpointing Mode
- Description
- The checkpointing mode (exactly-once vs. at-least-once).
- Related Name
-
execution.checkpointing.mode
- Default Value
- EXACTLY_ONCE
- API Name
-
checkpointing_mode
- Required
- true
Checkpointing Timeout (milliseconds)
- Description
- The maximum time that a checkpoint may take before being discarded.
- Related Name
-
execution.checkpointing.timeout
- Default Value
- 60000
- API Name
-
checkpointing_timeout
- Required
- true
Enable Hive Catalog for SQL Client
- Description
- Enables Hive Catalog for SQL Client
- Related Name
-
enable_hive_catalog
- Default Value
- false
- API Name
-
enable_hive_catalog
- Required
- false
JobManager JVM Options
- Description
- Java options to start the JVM of the JobManager with.
- Related Name
-
env.java.opts.jobmanager
- Default Value
- API Name
-
env_java_opts_jobmanager
- Required
- false
TaskManager JVM Options
- Description
- Java options to start the JVM of the TaskManager with.
- Related Name
-
env.java.opts.taskmanager
- Default Value
- API Name
-
env_java_opts_taskmanager
- Required
- false
Executor
- Description
- The name of the executor to be used for executing the given job.
- Related Name
-
execution.target
- Default Value
- yarn-per-job
- API Name
-
execution.target
- Required
- true
Network Buffer Timeout (milliseconds)
- Description
- The maximum time frequency (ms) for the flushing of the output buffers. By default the output buffers flush frequently to provide low latency and to aid smooth developer experience.
- Related Name
-
execution.buffer-timeout
- Default Value
- 100
- API Name
-
execution_buffer_timeout
- Required
- true
Enable Checkpoint Compression
- Description
- Tells if we should use compression for the state snapshot data or not.
- Related Name
-
execution.checkpointing.snapshot-compression
- Default Value
- false
- API Name
-
execution_snapshot_compression
- Required
- true
Externalized Checkpoint Retention
- Description
- The mode defines how an externalized checkpoint should be cleaned up on job cancellation. If you choose to retain externalized checkpoints on cancellation you have to handle checkpoint clean up manually when you cancel the job as well
- Related Name
-
execution.checkpointing.externalized-checkpoint-retention
- Default Value
- RETAIN_ON_CANCELLATION
- API Name
-
externalized_checkpoint_retention
- Required
- true
HDFS Service
- Description
- Name of the HDFS service that this Flink service instance depends on
- Related Name
- Default Value
- API Name
-
hdfs_service
- Required
- true
High Availability Service
- Description
- Defines high-availability mode used for the cluster execution. To enable high-availability, set this mode to 'ZOOKEEPER' or specify FQN of factory class.
- Related Name
-
high-availability
- Default Value
- ZOOKEEPER
- API Name
-
high_availability
- Required
- false
High Availability Storage Directory
- Description
- File system path (URI) where Flink persists metadata in high-availability setups. After changing this location please execute the `Create HA Directory` action of the Flink service.
- Related Name
-
high-availability.storageDir
- Default Value
- /user/flink/ha
- API Name
-
high_availability_storage_dir
- Required
- false
High Availability Zookeeper client ACL
- Description
- Defines the ACL (open|creator) to be configured on ZK node. The configuration value can be set to 'creator' if the ZooKeeper server configuration has the 'authProvider' property mapped to use SASLAuthenticationProvider and the cluster is configured to run in secure mode (Kerberos).
- Related Name
-
high-availability.zookeeper.client.acl
- Default Value
- open
- API Name
-
high_availability_zookeeper_client_acl
- Required
- false
CLI global dashboard fallback
- Description
- Allow the CLI to fall back to the global dashboard endpoint to access Flink jobs when the jobmanager address or yarn appId are not defined.
- Related Name
-
historyserver.cli.fallback
- Default Value
- true
- API Name
-
historyserver_cli_fallback
- Required
- false
Hive Service
- Description
- Name of the Hive service that this Flink service instance depends on
- Related Name
- Default Value
- API Name
-
hive_service
- Required
- false
JobManager Archive Directory (HDFS)
- Description
- Directory to upload completed jobs to. (Add this directory to the list of monitored directories of the HistoryServer as well.) After changing this location please execute the `Create JobManager Archive Directory` action of the Flink service.
- Related Name
-
jobmanager.archive.fs.dir
- Default Value
- /user/flink/applicationHistory
- API Name
-
jobmanager_archive_fs_dir
- Required
- true
JobManager Heap Size
- Description
- JVM heap size for the JobManager
- Related Name
-
jobmanager.heap.size
- Default Value
- 1 GiB
- API Name
-
jobmanager_heap_size
- Required
- true
Enable user specific archive subdirectory
- Description
- Boolean flag indicating whether the job archive should be written to a user specific subdirectory.
- Related Name
-
jobmanager.archive.per-user
- Default Value
- true
- API Name
-
jobmanager_per_user_archive
- Required
- false
Enable Kerberos Authentication
- Description
- Enables Kerberos authentication for Flink
- Related Name
-
kerberos.auth.enabled
- Default Value
- false
- API Name
-
kerberos.auth.enabled
- Required
- false
Max Concurrent Checkpoints
- Description
- The maximum number of checkpoint attempts that may be in progress at the same time. If this value is n, then no checkpoints will be triggered while n checkpoint attempts are currently in flight. For the next checkpoint to be triggered, one checkpoint attempt would need to finish or expire.
- Related Name
-
execution.checkpointing.max-concurrent-checkpoints
- Default Value
- 1
- API Name
-
max_concurrent_checkpoints
- Required
- true
Default Parallelism
- Description
- Default parallelism for jobs
- Related Name
-
parallelism.default
- Default Value
- 1
- API Name
-
parallelism_default
- Required
- true
Automatic Watermark Interval (milliseconds)
- Description
- The interval of the automatic watermark emission. Watermarks are used throughout the streaming system to keep track of the progress of time. They are used, for example, for time based windowing.
- Related Name
-
pipeline.auto-watermark-interval
- Default Value
- 200
- API Name
-
pipeline_auto_watermark_interval
- Required
- true
Allow Generic Types
- Description
- If the use of generic types is disabled, Flink will throw an UnsupportedOperationException whenever it encounters a data type that would go through Kryo for serialization.
- Related Name
-
pipeline.generic-types
- Default Value
- true
- API Name
-
pipeline_generic_types
- Required
- true
Max Parallelism
- Description
- The program-wide maximum parallelism used for operators which haven't specified a maximum parallelism. The maximum parallelism specifies the upper limit for dynamic scaling and the number of key groups used for partitioned state.
- Related Name
-
pipeline.max-parallelism
- Default Value
- API Name
-
pipeline_max_parallelism
- Required
- false
Enable Object Reuse
- Description
- When enabled objects that Flink internally uses for deserialization and passing data to user-code functions will be reused. Keep in mind that this can lead to bugs when the user-code function of an operation is not aware of this behaviour.
- Related Name
-
pipeline.object-reuse
- Default Value
- false
- API Name
-
pipeline_object_reuse
- Required
- true
Kerberos Login Contexts
- Description
- A comma-separated list of login contexts to provide the Kerberos credentials to (for example, `Client,KafkaClient` to use the credentials for ZooKeeper authentication and for Kafka authentication)
- Related Name
-
security.kerberos.login.contexts
- Default Value
- Client KafkaClient RegistryClient
- API Name
-
security_kerberos_login_contexts
- Required
- false
Kerberos Use Ticket Cache
- Description
- Indicates whether to read from your Kerberos ticket cache.
- Related Name
-
security.kerberos.login.use-ticket-cache
- Default Value
- true
- API Name
-
security_kerberos_login_use_ticket_cache
- Required
- false
Catalog for SQL Client
- Description
- Catalog for SQL Client
- Related Name
-
sql_current_catalog
- Default Value
- MEMORY
- API Name
-
sql_current_catalog
- Required
- true
SQL Client Current Database
- Description
- SQL Client Current Database
- Related Name
-
sql_current_db
- Default Value
- API Name
-
sql_current_db
- Required
- false
State Backend
- Description
- The state backend to be used to store and checkpoint state.
- Related Name
-
state.backend
- Default Value
- FILESYSTEM
- API Name
-
state_backend
- Required
- false
Incremental Checkpoints
- Description
- Option whether the state backend should create incremental checkpoints, if possible. For an incremental checkpoint, only a diff from the previous checkpoint is stored, rather than the complete checkpoint state. Some state backends may not support incremental checkpoints and ignore this option.
- Related Name
-
state.backend.incremental
- Default Value
- true
- API Name
-
state_backend_incremental
- Required
- true
Local State Recovery
- Description
- This option configures local recovery for this state backend. By default, local recovery is deactivated. Local recovery currently only covers keyed state backends. Currently, MemoryStateBackend does not support local recovery and ignore this option.
- Related Name
-
state.backend.local-recovery
- Default Value
- true
- API Name
-
state_backend_local_recovery
- Required
- true
RocksDB High-Prio Memory Fraction
- Description
- The fraction of cache memory that is reserved for high-priority data like index, filter, and compression dictionary blocks. This option only has an effect when 'state.backend.rocksdb.memory.managed' or 'state.backend.rocksdb.memory.fixed-per-slot' are configured.
- Related Name
-
state.backend.rocksdb.memory.high-prio-pool-ratio
- Default Value
- 0.1
- API Name
-
state_backend_rocksdb_memory_high_prio_ratio
- Required
- true
RocksDB Memory Management
- Description
- If set, the RocksDB state backend will automatically configure itself to use the managed memory budget of the task slot, and divide the memory over write buffers, indexes, block caches, etc. That way, the three major uses of memory of RocksDB will be capped.
- Related Name
-
state.backend.rocksdb.memory.managed
- Default Value
- true
- API Name
-
state_backend_rocksdb_memory_managed
- Required
- true
RocksDB Write Buffer Memory Fraction
- Description
- The maximum amount of memory that write buffers may take, as a fraction of the total shared memory. This option only has an effect when 'state.backend.rocksdb.memory.managed' or 'state.backend.rocksdb.memory.fixed-per-slot' are configured.
- Related Name
-
state.backend.rocksdb.memory.write-buffer-ratio
- Default Value
- 0.5
- API Name
-
state_backend_rocksdb_memory_write_buffer_ratio
- Required
- true
Predefined options for RocksDB state backend
- Description
- The predefined settings for RocksDB DBOptions and ColumnFamilyOptions by Flink community. Current supported candidate predefined-options are DEFAULT, SPINNING_DISK_OPTIMIZED, SPINNING_DISK_OPTIMIZED_HIGH_MEM or FLASH_SSD_OPTIMIZED. Note that user customized options and options from the RocksDBOptionsFactory are applied on top of these predefined ones.
- Related Name
-
state.backend.rocksdb.predefined-options
- Default Value
- DEFAULT
- API Name
-
state_backend_rocksdb_predefined_options
- Required
- true
RocksDB StateBackend Timer Service Factory
- Description
- This determines the factory for timer service state implementation. Options are either HEAP (heap-based, default) or ROCKSDB for an implementation based on RocksDB.
- Related Name
-
state.backend.rocksdb.timer-service.factory
- Default Value
- ROCKSDB
- API Name
-
state_backend_rocksdb_timer_service_factory
- Required
- true
State Checkpoints Directory (HDFS)
- Description
- The default directory used for storing the data files and meta data of checkpoints in a Flink supported filesystem. After changing this location please execute the `Create State Checkpoint Directory` action of the Flink service.
- Related Name
-
state.checkpoints.dir
- Default Value
- /user/flink/checkpoints
- API Name
-
state_checkpoints_dir
- Required
- false
Number of Checkpoints to Retain
- Description
- The maximum number of completed checkpoints to retain.
- Related Name
-
state.checkpoints.num-retained
- Default Value
- 3
- API Name
-
state_checkpoints_num_retained
- Required
- false
State Savepoints Directory (HDFS)
- Description
- The default directory used for storing the data files and meta data of checkpoints in a Flink supported filesystem. After changing this location please execute the `Create State Savepoint Directory` action of the Flink service.
- Related Name
-
state.savepoints.dir
- Default Value
- /user/flink/savepoints
- API Name
-
state_savepoints_dir
- Required
- false
TaskManager Managed Memory Fraction
- Description
- Fraction of Total Flink Memory to be used as Managed Memory, if Managed Memory size is not explicitly specified. Managed memory is used by Flink operators (caching, sorting, hashtables) and state backends (RocksDB).
- Related Name
-
taskmanager.memory.managed.fraction
- Default Value
- 0.4
- API Name
-
taskmanager_managed_memory_fraction
- Required
- true
TaskManager Process Memory Size
- Description
- This includes all the memory that a TaskExecutor consumes, consisting of Total Flink Memory, JVM Metaspace, and JVM Overhead. On containerized setups, this should be set to the container memory. See also 'taskmanager.memory.flink.size' for total Flink memory size configuration.
- Related Name
-
taskmanager.memory.process.size
- Default Value
- 2 GiB
- API Name
-
taskmanager_memory_process_size
- Required
- true
Network Buffer JVM Memory Fraction
- Description
- Fraction of JVM memory to use for network buffers. This determines how many streaming data exchange channels a TaskManager can have at the same time and how well buffered the channels are. If a job is rejected or you get a warning that the system has not enough buffers available, increase this value or the min/max values below. Also note, that 'taskmanager.memory.network.min' and 'taskmanager.memory.network.max' may override this fraction.
- Related Name
-
taskmanager.memory.network.fraction
- Default Value
- 0.1
- API Name
-
taskmanager_network_memory_fraction
- Required
- true
Network Buffer Memory Max
- Description
- Max Network Memory size for TaskExecutors. Network Memory is off-heap memory reserved for ShuffleEnvironment (e.g., network buffers). Network Memory size is derived to make up the configured fraction of the Total Flink Memory. If the derived size is less/greater than the configured min/max size, the min/max size will be used. The exact size of Network Memory can be explicitly specified by setting the min/max to the same value.
- Related Name
-
taskmanager.memory.network.max
- Default Value
- 2 GiB
- API Name
-
taskmanager_network_memory_max
- Required
- true
TaskManager Number of Task Slots
- Description
- The number of parallel operator or user function instances that a single TaskManager can run. If this value is larger than 1, a single TaskManager takes multiple instances of a function or operator. That way, the TaskManager can utilize multiple CPU cores, but at the same time, the available memory is divided between the different operator or function instances. This value is typically proportional to the number of physical CPU cores that the TaskManager's machine has (e.g., equal to the number of cores, or half the number of cores).
- Related Name
-
taskmanager.numberOfTaskSlots
- Default Value
- 1
- API Name
-
taskmanager_number_of_task_slots
- Required
- true
Maximum ApplicationMaster Attempts for YARN
- Description
- Number of ApplicationMaster restarts. Note that that the entire Flink cluster will restart and the YARN Client will loose the connection. Also, the JobManager address will change and you’ll need to set the JM host:port manually. It is recommended to leave this option at 1.
- Related Name
-
yarn.application-attempts
- Default Value
- 5
- API Name
-
yarn_application_attempts
- Required
- false
Maximum Failed Containers for YARN
- Description
- Maximum number of containers the system is going to reallocate in case of a failure.
- Related Name
-
yarn.maximum-failed-containers
- Default Value
- 100
- API Name
-
yarn_maximum_failed_containers
- Required
- false
YARN Service
- Description
- Name of the YARN service that this Flink service instance depends on
- Related Name
- Default Value
- API Name
-
yarn_service
- Required
- true
YARN Tags
- Description
- A comma-separated list of tags to apply to the Flink YARN application.
- Related Name
-
yarn.tags
- Default Value
- flink
- API Name
-
yarn_tags
- Required
- false
ZooKeeper Service
- Description
- Name of the ZooKeeper service that this Flink service instance depends on
- Related Name
- Default Value
- API Name
-
zookeeper_service
- Required
- true
Security
Kerberos Principal
- Description
- Kerberos principal short name used by all roles of this service.
- Related Name
- Default Value
- flink
- API Name
-
kerberos_princ_name
- Required
- true
Suppressions
Suppress Configuration Validator: CDH Version Validator
- Description
- Whether to suppress configuration warnings produced by the CDH Version Validator configuration validator.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_cdh_version_validator
- Required
- true
Suppress Configuration Validator: Deploy Directory
- Description
- Whether to suppress configuration warnings produced by the Deploy Directory configuration validator.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_client_config_root_dir
- Required
- true
Suppress Configuration Validator: Flink Client Advanced Configuration Snippet (Safety Valve) for flink-conf/atlas-application.properties
- Description
- Whether to suppress configuration warnings produced by the Flink Client Advanced Configuration Snippet (Safety Valve) for flink-conf/atlas-application.properties configuration validator.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_flink-conf/atlas-application.properties_client_config_safety_valve
- Required
- true
Suppress Configuration Validator: Flink Client Advanced Configuration Snippet (Safety Valve) for flink-conf/flink-conf.yaml
- Description
- Whether to suppress configuration warnings produced by the Flink Client Advanced Configuration Snippet (Safety Valve) for flink-conf/flink-conf.yaml configuration validator.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_flink-conf/flink-conf.yaml_client_config_safety_valve
- Required
- true
Suppress Configuration Validator: Flink Dashboard Advanced Configuration Snippet (Safety Valve) for flink-conf/flink-conf.yaml
- Description
- Whether to suppress configuration warnings produced by the Flink Dashboard Advanced Configuration Snippet (Safety Valve) for flink-conf/flink-conf.yaml configuration validator.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_flink-conf/flink-conf.yaml_role_safety_valve
- Required
- true
Suppress Configuration Validator: Flink Client Advanced Configuration Snippet (Safety Valve) for flink-conf/log4j-cli.properties
- Description
- Whether to suppress configuration warnings produced by the Flink Client Advanced Configuration Snippet (Safety Valve) for flink-conf/log4j-cli.properties configuration validator.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_flink-conf/log4j-cli.properties_client_config_safety_valve
- Required
- true
Suppress Configuration Validator: Flink Client Advanced Configuration Snippet (Safety Valve) for flink-conf/log4j.properties
- Description
- Whether to suppress configuration warnings produced by the Flink Client Advanced Configuration Snippet (Safety Valve) for flink-conf/log4j.properties configuration validator.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_flink-conf/log4j.properties_client_config_safety_valve
- Required
- true
Suppress Configuration Validator: Flink Client Advanced Configuration Snippet (Safety Valve) for flink-conf/sql-client-defaults.yaml
- Description
- Whether to suppress configuration warnings produced by the Flink Client Advanced Configuration Snippet (Safety Valve) for flink-conf/sql-client-defaults.yaml configuration validator.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_flink-conf/sql-client-defaults.yaml_client_config_safety_valve
- Required
- true
Suppress Configuration Validator: Flink Dashboard Environment Advanced Configuration Snippet (Safety Valve)
- Description
- Whether to suppress configuration warnings produced by the Flink Dashboard Environment Advanced Configuration Snippet (Safety Valve) configuration validator.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_flink_history_server_role_env_safety_valve
- Required
- true
Suppress Configuration Validator: History Server Archive Directory (HDFS)
- Description
- Whether to suppress configuration warnings produced by the History Server Archive Directory (HDFS) configuration validator.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_historyserver_archive_fs_dir
- Required
- true
Suppress Configuration Validator: Flink Dashboard Port
- Description
- Whether to suppress configuration warnings produced by the Flink Dashboard Port configuration validator.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_historyserver_web_port
- Required
- true
Suppress Configuration Validator: Flink Dashboard Logging Advanced Configuration Snippet (Safety Valve)
- Description
- Whether to suppress configuration warnings produced by the Flink Dashboard Logging Advanced Configuration Snippet (Safety Valve) configuration validator.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_log4j_safety_valve
- Required
- true
Suppress Configuration Validator: Flink Dashboard Log Directory
- Description
- Whether to suppress configuration warnings produced by the Flink Dashboard Log Directory configuration validator.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_log_dir
- Required
- true
Suppress Configuration Validator: Heap Dump Directory
- Description
- Whether to suppress configuration warnings produced by the Heap Dump Directory configuration validator.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_oom_heap_dump_dir
- Required
- true
Suppress Configuration Validator: Custom Control Group Resources (overrides Cgroup settings)
- Description
- Whether to suppress configuration warnings produced by the Custom Control Group Resources (overrides Cgroup settings) configuration validator.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_rm_custom_resources
- Required
- true
Suppress Configuration Validator: Role Triggers
- Description
- Whether to suppress configuration warnings produced by the Role Triggers configuration validator.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_role_triggers
- Required
- true
Suppress Configuration Validator: Kerberos Login Keytab
- Description
- Whether to suppress configuration warnings produced by the Kerberos Login Keytab configuration validator.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_security_kerberos_login_keytab
- Required
- true
Suppress Configuration Validator: Flink Dashboard TLS/SSL Trust Store File
- Description
- Whether to suppress configuration warnings produced by the Flink Dashboard TLS/SSL Trust Store File configuration validator.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_ssl_client_truststore_location
- Required
- true
Suppress Configuration Validator: Flink Dashboard TLS/SSL Trust Store Password
- Description
- Whether to suppress configuration warnings produced by the Flink Dashboard TLS/SSL Trust Store Password configuration validator.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_ssl_client_truststore_password
- Required
- true
Suppress Configuration Validator: Flink Dashboard TLS/SSL Server Keystore Key Password
- Description
- Whether to suppress configuration warnings produced by the Flink Dashboard TLS/SSL Server Keystore Key Password configuration validator.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_ssl_server_keystore_keypassword
- Required
- true
Suppress Configuration Validator: Flink Dashboard TLS/SSL Server Keystore File Location
- Description
- Whether to suppress configuration warnings produced by the Flink Dashboard TLS/SSL Server Keystore File Location configuration validator.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_ssl_server_keystore_location
- Required
- true
Suppress Configuration Validator: Flink Dashboard TLS/SSL Server Keystore File Password
- Description
- Whether to suppress configuration warnings produced by the Flink Dashboard TLS/SSL Server Keystore File Password configuration validator.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_ssl_server_keystore_password
- Required
- true
Suppress Configuration Validator: Stacks Collection Directory
- Description
- Whether to suppress configuration warnings produced by the Stacks Collection Directory configuration validator.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_stacks_collection_directory
- Required
- true
Suppress Parameter Validation: Atlas Metadata Namespace
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Atlas Metadata Namespace parameter.
- Related Name
- Default Value
- false
- API Name
-
service_config_suppression_atlas_metadata_namespace
- Required
- true
Suppress Parameter Validation: JobManager JVM Options
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the JobManager JVM Options parameter.
- Related Name
- Default Value
- false
- API Name
-
service_config_suppression_env_java_opts_jobmanager
- Required
- true
Suppress Parameter Validation: TaskManager JVM Options
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the TaskManager JVM Options parameter.
- Related Name
- Default Value
- false
- API Name
-
service_config_suppression_env_java_opts_taskmanager
- Required
- true
Suppress Parameter Validation: Flink Service Advanced Configuration Snippet (Safety Valve) for flink-conf/flink-conf.yaml
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Flink Service Advanced Configuration Snippet (Safety Valve) for flink-conf/flink-conf.yaml parameter.
- Related Name
- Default Value
- false
- API Name
-
service_config_suppression_flink-conf/flink-conf.yaml_service_safety_valve
- Required
- true
Suppress Configuration Validator: Flink Dashboard Count Validator
- Description
- Whether to suppress configuration warnings produced by the Flink Dashboard Count Validator configuration validator.
- Related Name
- Default Value
- false
- API Name
-
service_config_suppression_flink_history_server_count_validator
- Required
- true
Suppress Parameter Validation: Flink Service Environment Advanced Configuration Snippet (Safety Valve)
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Flink Service Environment Advanced Configuration Snippet (Safety Valve) parameter.
- Related Name
- Default Value
- false
- API Name
-
service_config_suppression_flink_service_env_safety_valve
- Required
- true
Suppress Configuration Validator: Gateway Count Validator
- Description
- Whether to suppress configuration warnings produced by the Gateway Count Validator configuration validator.
- Related Name
- Default Value
- false
- API Name
-
service_config_suppression_gateway_count_validator
- Required
- true
Suppress Parameter Validation: High Availability Storage Directory
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the High Availability Storage Directory parameter.
- Related Name
- Default Value
- false
- API Name
-
service_config_suppression_high_availability_storage_dir
- Required
- true
Suppress Parameter Validation: High Availability Zookeeper client ACL
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the High Availability Zookeeper client ACL parameter.
- Related Name
- Default Value
- false
- API Name
-
service_config_suppression_high_availability_zookeeper_client_acl
- Required
- true
Suppress Parameter Validation: JobManager Archive Directory (HDFS)
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the JobManager Archive Directory (HDFS) parameter.
- Related Name
- Default Value
- false
- API Name
-
service_config_suppression_jobmanager_archive_fs_dir
- Required
- true
Suppress Parameter Validation: Kerberos Principal
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Kerberos Principal parameter.
- Related Name
- Default Value
- false
- API Name
-
service_config_suppression_kerberos_princ_name
- Required
- true
Suppress Parameter Validation: System Group
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the System Group parameter.
- Related Name
- Default Value
- false
- API Name
-
service_config_suppression_process_groupname
- Required
- true
Suppress Parameter Validation: System User
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the System User parameter.
- Related Name
- Default Value
- false
- API Name
-
service_config_suppression_process_username
- Required
- true
Suppress Parameter Validation: Kerberos Login Contexts
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Kerberos Login Contexts parameter.
- Related Name
- Default Value
- false
- API Name
-
service_config_suppression_security_kerberos_login_contexts
- Required
- true
Suppress Parameter Validation: Service Triggers
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Service Triggers parameter.
- Related Name
- Default Value
- false
- API Name
-
service_config_suppression_service_triggers
- Required
- true
Suppress Parameter Validation: Service Monitor Derived Configs Advanced Configuration Snippet (Safety Valve)
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Service Monitor Derived Configs Advanced Configuration Snippet (Safety Valve) parameter.
- Related Name
- Default Value
- false
- API Name
-
service_config_suppression_smon_derived_configs_safety_valve
- Required
- true
Suppress Parameter Validation: SQL Client Current Database
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the SQL Client Current Database parameter.
- Related Name
- Default Value
- false
- API Name
-
service_config_suppression_sql_current_db
- Required
- true
Suppress Parameter Validation: State Checkpoints Directory (HDFS)
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the State Checkpoints Directory (HDFS) parameter.
- Related Name
- Default Value
- false
- API Name
-
service_config_suppression_state_checkpoints_dir
- Required
- true
Suppress Parameter Validation: State Savepoints Directory (HDFS)
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the State Savepoints Directory (HDFS) parameter.
- Related Name
- Default Value
- false
- API Name
-
service_config_suppression_state_savepoints_dir
- Required
- true
Suppress Parameter Validation: YARN Tags
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the YARN Tags parameter.
- Related Name
- Default Value
- false
- API Name
-
service_config_suppression_yarn_tags
- Required
- true