Sqoop Properties in Cloudera Runtime 7.2.x

Role groups:

Gateway

Advanced

Deploy Directory

Description: The directory where the client configs will be deployed
Related Name
Default Value: /etc/sqoop
API Name: client_config_root_dir
Required: true

Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/managers.d/cm_manager_overrides

Description: For advanced use only, a string to be inserted into the client configuration for sqoop-conf/managers.d/cm_manager_overrides.
Related Name
Default Value
API Name: sqoop-conf/managers.d/cm_manager_overrides_client_config_safety_valve
Required: false

Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-env.sh

Description: For advanced use only, a string to be inserted into the client configuration for sqoop-conf/sqoop-env.sh.
Related Name
Default Value
API Name: sqoop-conf/sqoop-env.sh_client_config_safety_valve
Required: false

Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-site.xml

Description: For advanced use only, a string to be inserted into the client configuration for sqoop-conf/sqoop-site.xml.
Related Name
Default Value
API Name: sqoop-conf/sqoop-site.xml_client_config_safety_valve
Required: false

Monitoring

Enable Configuration Change Alerts

Description: When set, Cloudera Manager will send alerts when this entity's configuration changes.
Related Name
Default Value: false
API Name: enable_config_alerts
Required: false

Other

Alternatives Priority

Description: The priority level that the client configuration will have in the Alternatives system on the hosts. Higher priority levels will cause Alternatives to prefer this configuration over any others.
Related Name
Default Value: 50
API Name: client_config_priority
Required: true

Sqoop connection factories

Description: A list of ManagerFactory implementations which are consulted, in order, to instantiate ConnManager instances used to drive connections to databases.
Related Name: sqoop.connection.factories
Default Value
API Name: sqoop_connection_factories
Required: false

Sqoop Tool Plugins

Description: A list of ToolPlugin implementations which are consulted, in order, to register SqoopTool instances which allow third-party tools to be used.
Related Name: sqoop.tool.plugins
Default Value
API Name: sqoop_tool_plugins
Required: false

Suppressions

Suppress Configuration Validator: CDH Version Validator

Description: Whether to suppress configuration warnings produced by the CDH Version Validator configuration validator.
Related Name
Default Value: false
API Name: role_config_suppression_cdh_version_validator
Required: true

Suppress Parameter Validation: Deploy Directory

Description: Whether to suppress configuration warnings produced by the built-in parameter validation for the Deploy Directory parameter.
Related Name
Default Value: false
API Name: role_config_suppression_client_config_root_dir
Required: true

Suppress Parameter Validation: Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/managers.d/cm_manager_overrides

Description: Whether to suppress configuration warnings produced by the built-in parameter validation for the Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/managers.d/cm_manager_overrides parameter.
Related Name
Default Value: false
API Name: role_config_suppression_sqoop-conf/managers.d/cm_manager_overrides_client_config_safety_valve
Required: true

Suppress Parameter Validation: Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-env.sh

Description: Whether to suppress configuration warnings produced by the built-in parameter validation for the Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-env.sh parameter.
Related Name
Default Value: false
API Name: role_config_suppression_sqoop-conf/sqoop-env.sh_client_config_safety_valve
Required: true

Suppress Parameter Validation: Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-site.xml

Description: Whether to suppress configuration warnings produced by the built-in parameter validation for the Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-site.xml parameter.
Related Name
Default Value: false
API Name: role_config_suppression_sqoop-conf/sqoop-site.xml_client_config_safety_valve
Required: true

Suppress Parameter Validation: Sqoop connection factories

Description: Whether to suppress configuration warnings produced by the built-in parameter validation for the Sqoop connection factories parameter.
Related Name
Default Value: false
API Name: role_config_suppression_sqoop_connection_factories
Required: true

Suppress Parameter Validation: Sqoop Tool Plugins

Description: Whether to suppress configuration warnings produced by the built-in parameter validation for the Sqoop Tool Plugins parameter.
Related Name
Default Value: false
API Name: role_config_suppression_sqoop_tool_plugins
Required: true

Service-Wide

Advanced

System Group

Description: The group that this service's processes should run as.
Related Name
Default Value: root
API Name: process_groupname
Required: true

System User

Description: The user that this service's processes should run as.
Related Name
Default Value: root
API Name: process_username
Required: true

Sqoop Service Environment Advanced Configuration Snippet (Safety Valve)

Description: For advanced use only, key-value pairs (one on each line) to be inserted into a role's environment. Applies to configurations of all roles in this service except client configuration.
Related Name
Default Value
API Name: SQOOP_CLIENT_service_env_safety_valve
Required: false

Monitoring

Enable Service Level Health Alerts

Description: When set, Cloudera Manager will send alerts when the health of this service reaches the threshold specified by the EventServer setting eventserver_health_events_alert_threshold
Related Name
Default Value: true
API Name: enable_alerts
Required: false

Enable Configuration Change Alerts

Description: When set, Cloudera Manager will send alerts when this entity's configuration changes.
Related Name
Default Value: false
API Name: enable_config_alerts
Required: false

Service Triggers

Description

The configured triggers for this service. This is a JSON-formatted list of triggers. These triggers are evaluated as part as the health system. Every trigger expression is parsed, and if the trigger condition is met, the list of actions provided in the trigger expression is executed. Each trigger has the following fields:

triggerName (mandatory) - The name of the trigger. This value must be unique for the specific service.
triggerExpression (mandatory) - A tsquery expression representing the trigger.
streamThreshold (optional) - The maximum number of streams that can satisfy a condition of a trigger before the condition fires. By default set to 0, and any stream returned causes the condition to fire.
enabled (optional) - By default set to 'true'. If set to 'false', the trigger is not evaluated.
expressionEditorConfig (optional) - Metadata for the trigger editor. If present, the trigger should only be edited from the Edit Trigger page; editing the trigger here can lead to inconsistencies.

For example, the following JSON formatted trigger fires if there are more than 10 DataNodes with more than 500 file descriptors opened:

[{"triggerName": "sample-trigger",
  "triggerExpression": "IF (SELECT fd_open WHERE roleType = DataNode and last(fd_open) > 500) DO health:bad",
  "streamThreshold": 10, "enabled": "true"}]

See the trigger rules documentation for more details on how to write triggers using tsquery.The JSON format is evolving and may change and, as a result, backward compatibility is not guaranteed between releases.

Related Name

Default Value

[]

API Name

service_triggers

Required

true

Service Monitor Derived Configs Advanced Configuration Snippet (Safety Valve)

Description: For advanced use only, a list of derived configuration properties that will be used by the Service Monitor instead of the default ones.
Related Name
Default Value
API Name: smon_derived_configs_safety_valve
Required: false

Other

Parquet writer implementation

Description: Configure the library used during parquet jobs for writing and reading data. Possible values: hadoop, kite (legacy).
Related Name: parquetjob.configurator.implementation
Default Value: hadoop
API Name: parquetjob.configurator.implementation
Required: false

Sqoop decimal padding

Description: Enables padding for fixed point number types (decimal, number, numeric) in case of Avro and Parquet imports.
Related Name: sqoop.avro.decimal_padding.enable
Default Value: true
API Name: sqoop.avro.decimal_padding.enable
Required: false

Default precision for logical types

Description: Specifies the default precision for fixed point number types. This value is only used if the precision was not specified in the source table.
Related Name: sqoop.avro.logical_types.decimal.default.precision
Default Value: 38
API Name: sqoop.avro.logical_types.decimal.default.precision
Required: false

Default scale for logical types

Description: Default scale for fixed point number types. This value is only used if the scale was not specified in the source table.
Related Name: sqoop.avro.logical_types.decimal.default.scale
Default Value: 10
API Name: sqoop.avro.logical_types.decimal.default.scale
Required: false

Enable avro logical types

Description: Enables the use of logical types in avro files, so that fixed point number types are converted to decimal during an import. Fixed point number types are converted to String if this property is set to false.
Related Name: sqoop.avro.logical_types.decimal.enable
Default Value: true
API Name: sqoop.avro.logical_types.decimal.enable
Required: false

Enable parquet logical types

Description: Enables the use of logical types in parquet files, so that fixed point number types are converted to decimal during an import. Fixed point number types are converted to String if this property is set to false.
Related Name: sqoop.parquet.logical_types.decimal.enable
Default Value: true
API Name: sqoop.parquet.logical_types.decimal.enable
Required: false

Suppressions

Suppress Configuration Validator: CDH Version Validator

Description: Whether to suppress configuration warnings produced by the CDH Version Validator configuration validator.
Related Name
Default Value: false
API Name: role_config_suppression_cdh_version_validator
Required: true

Suppress Configuration Validator: Deploy Directory

Description: Whether to suppress configuration warnings produced by the Deploy Directory configuration validator.
Related Name
Default Value: false
API Name: role_config_suppression_client_config_root_dir
Required: true

Suppress Configuration Validator: Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/managers.d/cm_manager_overrides

Description: Whether to suppress configuration warnings produced by the Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/managers.d/cm_manager_overrides configuration validator.
Related Name
Default Value: false
API Name: role_config_suppression_sqoop-conf/managers.d/cm_manager_overrides_client_config_safety_valve
Required: true

Suppress Configuration Validator: Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-env.sh

Description: Whether to suppress configuration warnings produced by the Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-env.sh configuration validator.
Related Name
Default Value: false
API Name: role_config_suppression_sqoop-conf/sqoop-env.sh_client_config_safety_valve
Required: true

Suppress Configuration Validator: Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-site.xml

Description: Whether to suppress configuration warnings produced by the Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-site.xml configuration validator.
Related Name
Default Value: false
API Name: role_config_suppression_sqoop-conf/sqoop-site.xml_client_config_safety_valve
Required: true

Suppress Configuration Validator: Sqoop connection factories

Description: Whether to suppress configuration warnings produced by the Sqoop connection factories configuration validator.
Related Name
Default Value: false
API Name: role_config_suppression_sqoop_connection_factories
Required: true

Suppress Configuration Validator: Sqoop Tool Plugins

Description: Whether to suppress configuration warnings produced by the Sqoop Tool Plugins configuration validator.
Related Name
Default Value: false
API Name: role_config_suppression_sqoop_tool_plugins
Required: true

Suppress Configuration Validator: Gateway Count Validator

Description: Whether to suppress configuration warnings produced by the Gateway Count Validator configuration validator.
Related Name
Default Value: false
API Name: service_config_suppression_gateway_count_validator
Required: true

Suppress Parameter Validation: Parquet writer implementation

Description: Whether to suppress configuration warnings produced by the built-in parameter validation for the Parquet writer implementation parameter.
Related Name
Default Value: false
API Name: service_config_suppression_parquetjob.configurator.implementation
Required: true

Suppress Parameter Validation: System Group

Description: Whether to suppress configuration warnings produced by the built-in parameter validation for the System Group parameter.
Related Name
Default Value: false
API Name: service_config_suppression_process_groupname
Required: true

Suppress Parameter Validation: System User

Description: Whether to suppress configuration warnings produced by the built-in parameter validation for the System User parameter.
Related Name
Default Value: false
API Name: service_config_suppression_process_username
Required: true

Suppress Parameter Validation: Service Triggers

Description: Whether to suppress configuration warnings produced by the built-in parameter validation for the Service Triggers parameter.
Related Name
Default Value: false
API Name: service_config_suppression_service_triggers
Required: true

Suppress Parameter Validation: Service Monitor Derived Configs Advanced Configuration Snippet (Safety Valve)

Description: Whether to suppress configuration warnings produced by the built-in parameter validation for the Service Monitor Derived Configs Advanced Configuration Snippet (Safety Valve) parameter.
Related Name
Default Value: false
API Name: service_config_suppression_smon_derived_configs_safety_valve
Required: true

Suppress Parameter Validation: Sqoop Service Environment Advanced Configuration Snippet (Safety Valve)

Description: Whether to suppress configuration warnings produced by the built-in parameter validation for the Sqoop Service Environment Advanced Configuration Snippet (Safety Valve) parameter.
Related Name
Default Value: false
API Name: service_config_suppression_sqoop_client_service_env_safety_valve
Required: true