Cloudera Docs

Sqoop Properties in CDH 7.0.2

Role groups:

Gateway

Categories:

Advanced

Display Name Description Related Name Default Value API Name Required
Deploy Directory The directory where the client configs will be deployed /etc/sqoop client_config_root_dir true
Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/managers.d/cm_manager_overrides For advanced use only, a string to be inserted into the client configuration for sqoop-conf/managers.d/cm_manager_overrides. sqoop-conf/managers.d/cm_manager_overrides_client_config_safety_valve false
Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-env.sh For advanced use only, a string to be inserted into the client configuration for sqoop-conf/sqoop-env.sh. sqoop-conf/sqoop-env.sh_client_config_safety_valve false
Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-site.xml For advanced use only, a string to be inserted into the client configuration for sqoop-conf/sqoop-site.xml. sqoop-conf/sqoop-site.xml_client_config_safety_valve false

Monitoring

Display Name Description Related Name Default Value API Name Required
Enable Configuration Change Alerts When set, Cloudera Manager will send alerts when this entity's configuration changes. false enable_config_alerts false

Other

Display Name Description Related Name Default Value API Name Required
Alternatives Priority The priority level that the client configuration will have in the Alternatives system on the hosts. Higher priority levels will cause Alternatives to prefer this configuration over any others. 50 client_config_priority true
Sqoop connection factories A list of ManagerFactory implementations which are consulted, in order, to instantiate ConnManager instances used to drive connections to databases. sqoop.connection.factories sqoop_connection_factories false
Sqoop Tool Plugins A list of ToolPlugin implementations which are consulted, in order, to register SqoopTool instances which allow third-party tools to be used. sqoop.tool.plugins sqoop_tool_plugins false

Suppressions

Display Name Description Related Name Default Value API Name Required
Suppress Configuration Validator: CDH Version Validator Whether to suppress configuration warnings produced by the CDH Version Validator configuration validator. false role_config_suppression_cdh_version_validator true
Suppress Parameter Validation: Deploy Directory Whether to suppress configuration warnings produced by the built-in parameter validation for the Deploy Directory parameter. false role_config_suppression_client_config_root_dir true
Suppress Parameter Validation: Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/managers.d/cm_manager_overrides Whether to suppress configuration warnings produced by the built-in parameter validation for the Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/managers.d/cm_manager_overrides parameter. false role_config_suppression_sqoop-conf/managers.d/cm_manager_overrides_client_config_safety_valve true
Suppress Parameter Validation: Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-env.sh Whether to suppress configuration warnings produced by the built-in parameter validation for the Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-env.sh parameter. false role_config_suppression_sqoop-conf/sqoop-env.sh_client_config_safety_valve true
Suppress Parameter Validation: Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-site.xml Whether to suppress configuration warnings produced by the built-in parameter validation for the Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-site.xml parameter. false role_config_suppression_sqoop-conf/sqoop-site.xml_client_config_safety_valve true
Suppress Parameter Validation: Sqoop connection factories Whether to suppress configuration warnings produced by the built-in parameter validation for the Sqoop connection factories parameter. false role_config_suppression_sqoop_connection_factories true
Suppress Parameter Validation: Sqoop Tool Plugins Whether to suppress configuration warnings produced by the built-in parameter validation for the Sqoop Tool Plugins parameter. false role_config_suppression_sqoop_tool_plugins true

Service-Wide

Categories:

Advanced

Display Name Description Related Name Default Value API Name Required
System Group The group that this service's processes should run as. root process_groupname true
System User The user that this service's processes should run as. root process_username true
Sqoop Service Environment Advanced Configuration Snippet (Safety Valve) For advanced use only, key-value pairs (one on each line) to be inserted into a role's environment. Applies to configurations of all roles in this service except client configuration. SQOOP_CLIENT_service_env_safety_valve false

Monitoring

Display Name Description Related Name Default Value API Name Required
Enable Service Level Health Alerts When set, Cloudera Manager will send alerts when the health of this service reaches the threshold specified by the EventServer setting eventserver_health_events_alert_threshold true enable_alerts false
Enable Configuration Change Alerts When set, Cloudera Manager will send alerts when this entity's configuration changes. false enable_config_alerts false
Service Triggers The configured triggers for this service. This is a JSON-formatted list of triggers. These triggers are evaluated as part as the health system. Every trigger expression is parsed, and if the trigger condition is met, the list of actions provided in the trigger expression is executed. Each trigger has the following fields:
  • triggerName (mandatory) - The name of the trigger. This value must be unique for the specific service.
  • triggerExpression (mandatory) - A tsquery expression representing the trigger.
  • streamThreshold (optional) - The maximum number of streams that can satisfy a condition of a trigger before the condition fires. By default set to 0, and any stream returned causes the condition to fire.
  • enabled (optional) - By default set to 'true'. If set to 'false', the trigger is not evaluated.
  • expressionEditorConfig (optional) - Metadata for the trigger editor. If present, the trigger should only be edited from the Edit Trigger page; editing the trigger here can lead to inconsistencies.
For example, the following JSON formatted trigger fires if there are more than 10 DataNodes with more than 500 file descriptors opened:[{"triggerName": "sample-trigger", "triggerExpression": "IF (SELECT fd_open WHERE roleType = DataNode and last(fd_open) > 500) DO health:bad", "streamThreshold": 10, "enabled": "true"}]See the trigger rules documentation for more details on how to write triggers using tsquery.The JSON format is evolving and may change and, as a result, backward compatibility is not guaranteed between releases.		 [] service_triggers true
Service Monitor Derived Configs Advanced Configuration Snippet (Safety Valve) For advanced use only, a list of derived configuration properties that will be used by the Service Monitor instead of the default ones. smon_derived_configs_safety_valve false

Other

Display Name Description Related Name Default Value API Name Required
Parquet writer implementation Configure the library used during parquet jobs for writing and reading data. Possible values: hadoop, kite (legacy). parquetjob.configurator.implementation hadoop parquetjob.configurator.implementation false
Sqoop decimal padding Enables padding for fixed point number types (decimal, number, numeric) in case of Avro and Parquet imports. sqoop.avro.decimal_padding.enable true sqoop.avro.decimal_padding.enable false
Default precision for logical types Specifies the default precision for fixed point number types. This value is only used if the precision was not specified in the source table. sqoop.avro.logical_types.decimal.default.precision 38 sqoop.avro.logical_types.decimal.default.precision false
Default scale for logical types Default scale for fixed point number types. This value is only used if the scale was not specified in the source table. sqoop.avro.logical_types.decimal.default.scale 10 sqoop.avro.logical_types.decimal.default.scale false
Enable avro logical types Enables the use of logical types in avro files, so that fixed point number types are converted to decimal during an import. Fixed point number types are converted to String if this property is set to false. sqoop.avro.logical_types.decimal.enable true sqoop.avro.logical_types.decimal.enable false
Enable parquet logical types Enables the use of logical types in parquet files, so that fixed point number types are converted to decimal during an import. Fixed point number types are converted to String if this property is set to false. sqoop.parquet.logical_types.decimal.enable true sqoop.parquet.logical_types.decimal.enable false

Suppressions

Display Name Description Related Name Default Value API Name Required
Suppress Configuration Validator: CDH Version Validator Whether to suppress configuration warnings produced by the CDH Version Validator configuration validator. false role_config_suppression_cdh_version_validator true
Suppress Configuration Validator: Deploy Directory Whether to suppress configuration warnings produced by the Deploy Directory configuration validator. false role_config_suppression_client_config_root_dir true
Suppress Configuration Validator: Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/managers.d/cm_manager_overrides Whether to suppress configuration warnings produced by the Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/managers.d/cm_manager_overrides configuration validator. false role_config_suppression_sqoop-conf/managers.d/cm_manager_overrides_client_config_safety_valve true
Suppress Configuration Validator: Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-env.sh Whether to suppress configuration warnings produced by the Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-env.sh configuration validator. false role_config_suppression_sqoop-conf/sqoop-env.sh_client_config_safety_valve true
Suppress Configuration Validator: Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-site.xml Whether to suppress configuration warnings produced by the Sqoop Client Advanced Configuration Snippet (Safety Valve) for sqoop-conf/sqoop-site.xml configuration validator. false role_config_suppression_sqoop-conf/sqoop-site.xml_client_config_safety_valve true
Suppress Configuration Validator: Sqoop connection factories Whether to suppress configuration warnings produced by the Sqoop connection factories configuration validator. false role_config_suppression_sqoop_connection_factories true
Suppress Configuration Validator: Sqoop Tool Plugins Whether to suppress configuration warnings produced by the Sqoop Tool Plugins configuration validator. false role_config_suppression_sqoop_tool_plugins true
Suppress Configuration Validator: Gateway Count Validator Whether to suppress configuration warnings produced by the Gateway Count Validator configuration validator. false service_config_suppression_gateway_count_validator true
Suppress Parameter Validation: Parquet writer implementation Whether to suppress configuration warnings produced by the built-in parameter validation for the Parquet writer implementation parameter. false service_config_suppression_parquetjob.configurator.implementation true
Suppress Parameter Validation: System Group Whether to suppress configuration warnings produced by the built-in parameter validation for the System Group parameter. false service_config_suppression_process_groupname true
Suppress Parameter Validation: System User Whether to suppress configuration warnings produced by the built-in parameter validation for the System User parameter. false service_config_suppression_process_username true
Suppress Parameter Validation: Service Triggers Whether to suppress configuration warnings produced by the built-in parameter validation for the Service Triggers parameter. false service_config_suppression_service_triggers true
Suppress Parameter Validation: Service Monitor Derived Configs Advanced Configuration Snippet (Safety Valve) Whether to suppress configuration warnings produced by the built-in parameter validation for the Service Monitor Derived Configs Advanced Configuration Snippet (Safety Valve) parameter. false service_config_suppression_smon_derived_configs_safety_valve true
Suppress Parameter Validation: Sqoop Service Environment Advanced Configuration Snippet (Safety Valve) Whether to suppress configuration warnings produced by the built-in parameter validation for the Sqoop Service Environment Advanced Configuration Snippet (Safety Valve) parameter. false service_config_suppression_sqoop_client_service_env_safety_valve true