Sqoop Properties in Cloudera Runtime 7.2.x

Role groups:

Gateway

Advanced

Deploy Directory

Description
The directory where the client configs will be deployed
Related Name
Default Value
/etc/sqoop
API Name
client_config_root_dir
Required
true

Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/managers.d/cm_manager_overrides

Description
For advanced use only, a string to be inserted into the client configuration for sqoop-conf/managers.d/cm_manager_overrides.
Related Name
Default Value
API Name
sqoop-conf/managers.d/cm_manager_overrides_client_config_safety_valve
Required
false

Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-env.sh

Description
For advanced use only, a string to be inserted into the client configuration for sqoop-conf/sqoop-env.sh.
Related Name
Default Value
API Name
sqoop-conf/sqoop-env.sh_client_config_safety_valve
Required
false

Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-site.xml

Description
For advanced use only, a string to be inserted into the client configuration for sqoop-conf/sqoop-site.xml.
Related Name
Default Value
API Name
sqoop-conf/sqoop-site.xml_client_config_safety_valve
Required
false

Monitoring

Enable Configuration Change Alerts

Description
When set, Cloudera Manager will send alerts when this entity's configuration changes.
Related Name
Default Value
false
API Name
enable_config_alerts
Required
false

Other

Alternatives Priority

Description
The priority level that the client configuration will have in the Alternatives system on the hosts. Higher priority levels will cause Alternatives to prefer this configuration over any others.
Related Name
Default Value
50
API Name
client_config_priority
Required
true

Sqoop connection factories

Description
A list of ManagerFactory implementations which are consulted, in order, to instantiate ConnManager instances used to drive connections to databases.
Related Name
sqoop.connection.factories
Default Value
API Name
sqoop_connection_factories
Required
false

Sqoop Tool Plugins

Description
A list of ToolPlugin implementations which are consulted, in order, to register SqoopTool instances which allow third-party tools to be used.
Related Name
sqoop.tool.plugins
Default Value
API Name
sqoop_tool_plugins
Required
false

Suppressions

Suppress Configuration Validator: CDH Version Validator

Description
Whether to suppress configuration warnings produced by the CDH Version Validator configuration validator.
Related Name
Default Value
false
API Name
role_config_suppression_cdh_version_validator
Required
true

Suppress Parameter Validation: Deploy Directory

Description
Whether to suppress configuration warnings produced by the built-in parameter validation for the Deploy Directory parameter.
Related Name
Default Value
false
API Name
role_config_suppression_client_config_root_dir
Required
true

Suppress Parameter Validation: Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/managers.d/cm_manager_overrides

Description
Whether to suppress configuration warnings produced by the built-in parameter validation for the Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/managers.d/cm_manager_overrides parameter.
Related Name
Default Value
false
API Name
role_config_suppression_sqoop-conf/managers.d/cm_manager_overrides_client_config_safety_valve
Required
true

Suppress Parameter Validation: Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-env.sh

Description
Whether to suppress configuration warnings produced by the built-in parameter validation for the Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-env.sh parameter.
Related Name
Default Value
false
API Name
role_config_suppression_sqoop-conf/sqoop-env.sh_client_config_safety_valve
Required
true

Suppress Parameter Validation: Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-site.xml

Description
Whether to suppress configuration warnings produced by the built-in parameter validation for the Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-site.xml parameter.
Related Name
Default Value
false
API Name
role_config_suppression_sqoop-conf/sqoop-site.xml_client_config_safety_valve
Required
true

Suppress Parameter Validation: Sqoop connection factories

Description
Whether to suppress configuration warnings produced by the built-in parameter validation for the Sqoop connection factories parameter.
Related Name
Default Value
false
API Name
role_config_suppression_sqoop_connection_factories
Required
true

Suppress Parameter Validation: Sqoop Tool Plugins

Description
Whether to suppress configuration warnings produced by the built-in parameter validation for the Sqoop Tool Plugins parameter.
Related Name
Default Value
false
API Name
role_config_suppression_sqoop_tool_plugins
Required
true

Service-Wide

Advanced

System Group

Description
The group that this service's processes should run as.
Related Name
Default Value
root
API Name
process_groupname
Required
true

System User

Description
The user that this service's processes should run as.
Related Name
Default Value
root
API Name
process_username
Required
true

Sqoop Service Environment Advanced Configuration Snippet (Safety Valve)

Description
For advanced use only, key-value pairs (one on each line) to be inserted into a role's environment. Applies to configurations of all roles in this service except client configuration.
Related Name
Default Value
API Name
SQOOP_CLIENT_service_env_safety_valve
Required
false

Monitoring

Enable Service Level Health Alerts

Description
When set, Cloudera Manager will send alerts when the health of this service reaches the threshold specified by the EventServer setting eventserver_health_events_alert_threshold
Related Name
Default Value
true
API Name
enable_alerts
Required
false

Enable Configuration Change Alerts

Description
When set, Cloudera Manager will send alerts when this entity's configuration changes.
Related Name
Default Value
false
API Name
enable_config_alerts
Required
false

Service Triggers

Description
The configured triggers for this service. This is a JSON-formatted list of triggers. These triggers are evaluated as part as the health system. Every trigger expression is parsed, and if the trigger condition is met, the list of actions provided in the trigger expression is executed. Each trigger has the following fields:
  • triggerName (mandatory) - The name of the trigger. This value must be unique for the specific service.
  • triggerExpression (mandatory) - A tsquery expression representing the trigger.
  • streamThreshold (optional) - The maximum number of streams that can satisfy a condition of a trigger before the condition fires. By default set to 0, and any stream returned causes the condition to fire.
  • enabled (optional) - By default set to 'true'. If set to 'false', the trigger is not evaluated.
  • expressionEditorConfig (optional) - Metadata for the trigger editor. If present, the trigger should only be edited from the Edit Trigger page; editing the trigger here can lead to inconsistencies.
For example, the following JSON formatted trigger fires if there are more than 10 DataNodes with more than 500 file descriptors opened:[{"triggerName": "sample-trigger", "triggerExpression": "IF (SELECT fd_open WHERE roleType = DataNode and last(fd_open) > 500) DO health:bad", "streamThreshold": 10, "enabled": "true"}]See the trigger rules documentation for more details on how to write triggers using tsquery.The JSON format is evolving and may change and, as a result, backward compatibility is not guaranteed between releases.
Related Name
Default Value
[]
API Name
service_triggers
Required
true

Service Monitor Derived Configs Advanced Configuration Snippet (Safety Valve)

Description
For advanced use only, a list of derived configuration properties that will be used by the Service Monitor instead of the default ones.
Related Name
Default Value
API Name
smon_derived_configs_safety_valve
Required
false

Other

Parquet writer implementation

Description
Configure the library used during parquet jobs for writing and reading data. Possible values: hadoop, kite (legacy).
Related Name
parquetjob.configurator.implementation
Default Value
hadoop
API Name
parquetjob.configurator.implementation
Required
false

Sqoop decimal padding

Description
Enables padding for fixed point number types (decimal, number, numeric) in case of Avro and Parquet imports.
Related Name
sqoop.avro.decimal_padding.enable
Default Value
true
API Name
sqoop.avro.decimal_padding.enable
Required
false

Default precision for logical types

Description
Specifies the default precision for fixed point number types. This value is only used if the precision was not specified in the source table.
Related Name
sqoop.avro.logical_types.decimal.default.precision
Default Value
38
API Name
sqoop.avro.logical_types.decimal.default.precision
Required
false

Default scale for logical types

Description
Default scale for fixed point number types. This value is only used if the scale was not specified in the source table.
Related Name
sqoop.avro.logical_types.decimal.default.scale
Default Value
10
API Name
sqoop.avro.logical_types.decimal.default.scale
Required
false

Enable avro logical types

Description
Enables the use of logical types in avro files, so that fixed point number types are converted to decimal during an import. Fixed point number types are converted to String if this property is set to false.
Related Name
sqoop.avro.logical_types.decimal.enable
Default Value
true
API Name
sqoop.avro.logical_types.decimal.enable
Required
false

Enable parquet logical types

Description
Enables the use of logical types in parquet files, so that fixed point number types are converted to decimal during an import. Fixed point number types are converted to String if this property is set to false.
Related Name
sqoop.parquet.logical_types.decimal.enable
Default Value
true
API Name
sqoop.parquet.logical_types.decimal.enable
Required
false

Suppressions

Suppress Configuration Validator: CDH Version Validator

Description
Whether to suppress configuration warnings produced by the CDH Version Validator configuration validator.
Related Name
Default Value
false
API Name
role_config_suppression_cdh_version_validator
Required
true

Suppress Configuration Validator: Deploy Directory

Description
Whether to suppress configuration warnings produced by the Deploy Directory configuration validator.
Related Name
Default Value
false
API Name
role_config_suppression_client_config_root_dir
Required
true

Suppress Configuration Validator: Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/managers.d/cm_manager_overrides

Description
Whether to suppress configuration warnings produced by the Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/managers.d/cm_manager_overrides configuration validator.
Related Name
Default Value
false
API Name
role_config_suppression_sqoop-conf/managers.d/cm_manager_overrides_client_config_safety_valve
Required
true

Suppress Configuration Validator: Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-env.sh

Description
Whether to suppress configuration warnings produced by the Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-env.sh configuration validator.
Related Name
Default Value
false
API Name
role_config_suppression_sqoop-conf/sqoop-env.sh_client_config_safety_valve
Required
true

Suppress Configuration Validator: Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-site.xml

Description
Whether to suppress configuration warnings produced by the Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-site.xml configuration validator.
Related Name
Default Value
false
API Name
role_config_suppression_sqoop-conf/sqoop-site.xml_client_config_safety_valve
Required
true

Suppress Configuration Validator: Sqoop connection factories

Description
Whether to suppress configuration warnings produced by the Sqoop connection factories configuration validator.
Related Name
Default Value
false
API Name
role_config_suppression_sqoop_connection_factories
Required
true

Suppress Configuration Validator: Sqoop Tool Plugins

Description
Whether to suppress configuration warnings produced by the Sqoop Tool Plugins configuration validator.
Related Name
Default Value
false
API Name
role_config_suppression_sqoop_tool_plugins
Required
true

Suppress Configuration Validator: Gateway Count Validator

Description
Whether to suppress configuration warnings produced by the Gateway Count Validator configuration validator.
Related Name
Default Value
false
API Name
service_config_suppression_gateway_count_validator
Required
true

Suppress Parameter Validation: Parquet writer implementation

Description
Whether to suppress configuration warnings produced by the built-in parameter validation for the Parquet writer implementation parameter.
Related Name
Default Value
false
API Name
service_config_suppression_parquetjob.configurator.implementation
Required
true

Suppress Parameter Validation: System Group

Description
Whether to suppress configuration warnings produced by the built-in parameter validation for the System Group parameter.
Related Name
Default Value
false
API Name
service_config_suppression_process_groupname
Required
true

Suppress Parameter Validation: System User

Description
Whether to suppress configuration warnings produced by the built-in parameter validation for the System User parameter.
Related Name
Default Value
false
API Name
service_config_suppression_process_username
Required
true

Suppress Parameter Validation: Service Triggers

Description
Whether to suppress configuration warnings produced by the built-in parameter validation for the Service Triggers parameter.
Related Name
Default Value
false
API Name
service_config_suppression_service_triggers
Required
true

Suppress Parameter Validation: Service Monitor Derived Configs Advanced Configuration Snippet (Safety Valve)

Description
Whether to suppress configuration warnings produced by the built-in parameter validation for the Service Monitor Derived Configs Advanced Configuration Snippet (Safety Valve) parameter.
Related Name
Default Value
false
API Name
service_config_suppression_smon_derived_configs_safety_valve
Required
true

Suppress Parameter Validation: Sqoop Service Environment Advanced Configuration Snippet (Safety Valve)

Description
Whether to suppress configuration warnings produced by the built-in parameter validation for the Sqoop Service Environment Advanced Configuration Snippet (Safety Valve) parameter.
Related Name
Default Value
false
API Name
service_config_suppression_sqoop_client_service_env_safety_valve
Required
true