Sqoop Properties in Cloudera Runtime 7.1
Role groups:
Gateway
Advanced
Deploy Directory
- Description
- The directory where the client configs will be deployed
- Related Name
- Default Value
- /etc/sqoop
- API Name
-
client_config_root_dir
- Required
- true
Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/managers.d/cm_manager_overrides
- Description
- For advanced use only, a string to be inserted into the client configuration for sqoop-conf/managers.d/cm_manager_overrides.
- Related Name
- Default Value
- API Name
-
sqoop-conf/managers.d/cm_manager_overrides_client_config_safety_valve
- Required
- false
Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-env.sh
- Description
- For advanced use only, a string to be inserted into the client configuration for sqoop-conf/sqoop-env.sh.
- Related Name
- Default Value
- API Name
-
sqoop-conf/sqoop-env.sh_client_config_safety_valve
- Required
- false
Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-site.xml
- Description
- For advanced use only, a string to be inserted into the client configuration for sqoop-conf/sqoop-site.xml.
- Related Name
- Default Value
- API Name
-
sqoop-conf/sqoop-site.xml_client_config_safety_valve
- Required
- false
Monitoring
Enable Configuration Change Alerts
- Description
- When set, Cloudera Manager will send alerts when this entity's configuration changes.
- Related Name
- Default Value
- false
- API Name
-
enable_config_alerts
- Required
- false
Other
Alternatives Priority
- Description
- The priority level that the client configuration will have in the Alternatives system on the hosts. Higher priority levels will cause Alternatives to prefer this configuration over any others.
- Related Name
- Default Value
- 50
- API Name
-
client_config_priority
- Required
- true
Sqoop connection factories
- Description
- A list of ManagerFactory implementations which are consulted, in order, to instantiate ConnManager instances used to drive connections to databases.
- Related Name
-
sqoop.connection.factories
- Default Value
- API Name
-
sqoop_connection_factories
- Required
- false
Sqoop Tool Plugins
- Description
- A list of ToolPlugin implementations which are consulted, in order, to register SqoopTool instances which allow third-party tools to be used.
- Related Name
-
sqoop.tool.plugins
- Default Value
- API Name
-
sqoop_tool_plugins
- Required
- false
Suppressions
Suppress Configuration Validator: CDH Version Validator
- Description
- Whether to suppress configuration warnings produced by the CDH Version Validator configuration validator.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_cdh_version_validator
- Required
- true
Suppress Parameter Validation: Deploy Directory
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Deploy Directory parameter.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_client_config_root_dir
- Required
- true
Suppress Parameter Validation: Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/managers.d/cm_manager_overrides
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/managers.d/cm_manager_overrides parameter.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_sqoop-conf/managers.d/cm_manager_overrides_client_config_safety_valve
- Required
- true
Suppress Parameter Validation: Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-env.sh
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-env.sh parameter.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_sqoop-conf/sqoop-env.sh_client_config_safety_valve
- Required
- true
Suppress Parameter Validation: Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-site.xml
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-site.xml parameter.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_sqoop-conf/sqoop-site.xml_client_config_safety_valve
- Required
- true
Suppress Parameter Validation: Sqoop connection factories
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Sqoop connection factories parameter.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_sqoop_connection_factories
- Required
- true
Suppress Parameter Validation: Sqoop Tool Plugins
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Sqoop Tool Plugins parameter.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_sqoop_tool_plugins
- Required
- true
Service-Wide
Advanced
System Group
- Description
- The group that this service's processes should run as.
- Related Name
- Default Value
- root
- API Name
-
process_groupname
- Required
- true
System User
- Description
- The user that this service's processes should run as.
- Related Name
- Default Value
- root
- API Name
-
process_username
- Required
- true
Sqoop Service Environment Advanced Configuration Snippet (Safety Valve)
- Description
- For advanced use only, key-value pairs (one on each line) to be inserted into a role's environment. Applies to configurations of all roles in this service except client configuration.
- Related Name
- Default Value
- API Name
-
SQOOP_CLIENT_service_env_safety_valve
- Required
- false
Monitoring
Enable Service Level Health Alerts
- Description
- When set, Cloudera Manager will send alerts when the health of this service reaches the threshold specified by the EventServer setting eventserver_health_events_alert_threshold
- Related Name
- Default Value
- true
- API Name
-
enable_alerts
- Required
- false
Enable Configuration Change Alerts
- Description
- When set, Cloudera Manager will send alerts when this entity's configuration changes.
- Related Name
- Default Value
- false
- API Name
-
enable_config_alerts
- Required
- false
Service Triggers
- Description
- The configured triggers for this service. This is a JSON-formatted list of triggers. These triggers are evaluated as part as the health system. Every trigger expression is parsed, and if the trigger condition is met, the list of actions provided in the trigger expression is executed. Each trigger has the following fields:
triggerName
(mandatory) - The name of the trigger. This value must be unique for the specific service.triggerExpression
(mandatory) - A tsquery expression representing the trigger.streamThreshold
(optional) - The maximum number of streams that can satisfy a condition of a trigger before the condition fires. By default set to 0, and any stream returned causes the condition to fire.enabled
(optional) - By default set to 'true'. If set to 'false', the trigger is not evaluated.expressionEditorConfig
(optional) - Metadata for the trigger editor. If present, the trigger should only be edited from the Edit Trigger page; editing the trigger here can lead to inconsistencies.
[{"triggerName": "sample-trigger", "triggerExpression": "IF (SELECT fd_open WHERE roleType = DataNode and last(fd_open) > 500) DO health:bad", "streamThreshold": 10, "enabled": "true"}]
See the trigger rules documentation for more details on how to write triggers using tsquery.The JSON format is evolving and may change and, as a result, backward compatibility is not guaranteed between releases. - Related Name
- Default Value
- []
- API Name
-
service_triggers
- Required
- true
Service Monitor Derived Configs Advanced Configuration Snippet (Safety Valve)
- Description
- For advanced use only, a list of derived configuration properties that will be used by the Service Monitor instead of the default ones.
- Related Name
- Default Value
- API Name
-
smon_derived_configs_safety_valve
- Required
- false
Other
Parquet writer implementation
- Description
- Configure the library used during parquet jobs for writing and reading data. Possible values: hadoop, kite (legacy).
- Related Name
-
parquetjob.configurator.implementation
- Default Value
- hadoop
- API Name
-
parquetjob.configurator.implementation
- Required
- false
Sqoop decimal padding
- Description
- Enables padding for fixed point number types (decimal, number, numeric) in case of Avro and Parquet imports.
- Related Name
-
sqoop.avro.decimal_padding.enable
- Default Value
- true
- API Name
-
sqoop.avro.decimal_padding.enable
- Required
- false
Default precision for logical types
- Description
- Specifies the default precision for fixed point number types. This value is only used if the precision was not specified in the source table.
- Related Name
-
sqoop.avro.logical_types.decimal.default.precision
- Default Value
- 38
- API Name
-
sqoop.avro.logical_types.decimal.default.precision
- Required
- false
Default scale for logical types
- Description
- Default scale for fixed point number types. This value is only used if the scale was not specified in the source table.
- Related Name
-
sqoop.avro.logical_types.decimal.default.scale
- Default Value
- 10
- API Name
-
sqoop.avro.logical_types.decimal.default.scale
- Required
- false
Enable avro logical types
- Description
- Enables the use of logical types in avro files, so that fixed point number types are converted to decimal during an import. Fixed point number types are converted to String if this property is set to false.
- Related Name
-
sqoop.avro.logical_types.decimal.enable
- Default Value
- true
- API Name
-
sqoop.avro.logical_types.decimal.enable
- Required
- false
Enable parquet logical types
- Description
- Enables the use of logical types in parquet files, so that fixed point number types are converted to decimal during an import. Fixed point number types are converted to String if this property is set to false.
- Related Name
-
sqoop.parquet.logical_types.decimal.enable
- Default Value
- true
- API Name
-
sqoop.parquet.logical_types.decimal.enable
- Required
- false
Suppressions
Suppress Configuration Validator: CDH Version Validator
- Description
- Whether to suppress configuration warnings produced by the CDH Version Validator configuration validator.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_cdh_version_validator
- Required
- true
Suppress Configuration Validator: Deploy Directory
- Description
- Whether to suppress configuration warnings produced by the Deploy Directory configuration validator.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_client_config_root_dir
- Required
- true
Suppress Configuration Validator: Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/managers.d/cm_manager_overrides
- Description
- Whether to suppress configuration warnings produced by the Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/managers.d/cm_manager_overrides configuration validator.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_sqoop-conf/managers.d/cm_manager_overrides_client_config_safety_valve
- Required
- true
Suppress Configuration Validator: Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-env.sh
- Description
- Whether to suppress configuration warnings produced by the Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-env.sh configuration validator.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_sqoop-conf/sqoop-env.sh_client_config_safety_valve
- Required
- true
Suppress Configuration Validator: Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-site.xml
- Description
- Whether to suppress configuration warnings produced by the Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-site.xml configuration validator.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_sqoop-conf/sqoop-site.xml_client_config_safety_valve
- Required
- true
Suppress Configuration Validator: Sqoop connection factories
- Description
- Whether to suppress configuration warnings produced by the Sqoop connection factories configuration validator.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_sqoop_connection_factories
- Required
- true
Suppress Configuration Validator: Sqoop Tool Plugins
- Description
- Whether to suppress configuration warnings produced by the Sqoop Tool Plugins configuration validator.
- Related Name
- Default Value
- false
- API Name
-
role_config_suppression_sqoop_tool_plugins
- Required
- true
Suppress Configuration Validator: Gateway Count Validator
- Description
- Whether to suppress configuration warnings produced by the Gateway Count Validator configuration validator.
- Related Name
- Default Value
- false
- API Name
-
service_config_suppression_gateway_count_validator
- Required
- true
Suppress Parameter Validation: Parquet writer implementation
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Parquet writer implementation parameter.
- Related Name
- Default Value
- false
- API Name
-
service_config_suppression_parquetjob.configurator.implementation
- Required
- true
Suppress Parameter Validation: System Group
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the System Group parameter.
- Related Name
- Default Value
- false
- API Name
-
service_config_suppression_process_groupname
- Required
- true
Suppress Parameter Validation: System User
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the System User parameter.
- Related Name
- Default Value
- false
- API Name
-
service_config_suppression_process_username
- Required
- true
Suppress Parameter Validation: Service Triggers
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Service Triggers parameter.
- Related Name
- Default Value
- false
- API Name
-
service_config_suppression_service_triggers
- Required
- true
Suppress Parameter Validation: Service Monitor Derived Configs Advanced Configuration Snippet (Safety Valve)
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Service Monitor Derived Configs Advanced Configuration Snippet (Safety Valve) parameter.
- Related Name
- Default Value
- false
- API Name
-
service_config_suppression_smon_derived_configs_safety_valve
- Required
- true
Suppress Parameter Validation: Sqoop Service Environment Advanced Configuration Snippet (Safety Valve)
- Description
- Whether to suppress configuration warnings produced by the built-in parameter validation for the Sqoop Service Environment Advanced Configuration Snippet (Safety Valve) parameter.
- Related Name
- Default Value
- false
- API Name
-
service_config_suppression_sqoop_client_service_env_safety_valve
- Required
- true