Fixed Issues in Apache Sqoop

Review the list of Sqoop issues that are resolved in Cloudera Runtime 7.2.10.

CDPD-15750: Sqoop commands should be executed on the cluster instead of the system-test container.

This issue is now resolved.

CDPQE-5981: PostgreSQL authentication failed for user "sqoozie" on AWS HA cluster.

This issue is now resolved.

CDPQE-5994: Connection to Azure postgres server fails due to incorrectly specified username to psql client.

This issue is now resolved.

CDPD-24462: Sqoop does not close DB connections in all cases

In certain scenarios Sqoop left database connections open. Handling is now safer and fixed the underlying issues. This issue is now resolved.

CDPD-24825: getPrimaryKeyQuery returned the columns in a non-deterministic order.

Fixed a bug where getPrimaryKeyQuery returned the columns in a non-deterministic order. This issue is now resolved.

CDPD-20444: The Sqoop build no longer shades the Avro and Parquet libraries as it wasn't needed for a long time.

Oozie now automatically pulls in the necessary avro and parquet libraries into Oozie's Sqoop sharelib. If Avro or Parquet is used with Sqoop with an Oozie Sqoop action then you need not copy these libraries to sharelib manually. This issue is now resolved.

CDPD-23157: Sqoop Teradata import fails if source table is empty.

This issue is now resolved.

CDPD-19934: Sqoop should handle keeping custom environment variables when executing beeline in a new process

Until now during Hive import when no --hs2-url parameter was specified and Sqoop did the import via the beeline command line utility, in the beeline process launched by Sqoop only the following environment variables were preserved from the parent process: HADOOP_COMMON_HOME, HADOOP_HOME, HADOOP_MAPRED_HOME, HBASE_HOME, HCAT_HOME, HIVE_HOME, JAVA_HOME, PATH, ZOOKEEPER_HOME. Now you can specify custom environment variables to preserve via:

• By specify the "sqoop.beeline.env.preserve" system-property for the Sqoop command. E.g.: sqoop import -Dsqoop.beeline.env.preserve=MY_VARIBALE
• Or by specify the "sqoop.beeline.env.preserve" property as a sqoop-site.xml safety-valve in Cloudera Manager and then it will be applied for every sqoop Hive import.

This issue is now resolved.

CDPD-21148: Remove Accumulo dependency from Sqoop

Accumulo support is removed from Sqoop. This issue is now resolved.

CDPD-18796: In Cloudera Manager, if sqoop.avro.logical_types.decimal.default.precision and sqoop.avro.logical_types.decimal.default.scale are not set correctly for the Sqoop component, the import job fails with the error "Error: java.lang.ArithmeticException: Rounding necessary".

This issue is now resolved.

CDPD-19647: When the --query parameter is used with --as-orcfile, Sqoop displays an error.

This issue is now resolved.

CDPD-17306: Hive-Common is added as a dependency to Sqoop and Oozie's Sqoop sharelib so that you do not have to do it manually.

This issue is now resolved.

CDPD-17843: Hive-JDBC is added as a dependency to Sqoop and Oozie's Sqoop sharelib so you do not have to do it manually.

This issue is now resolved.

CDPD-12646: Sqoop does not close the open database connection before submitting the MapReduce Job. The open connection utilizes resources and displays an error message in the log when the connection times out.

This issue is resolved.