Fixed Issues in Cloud Connectors

Review the list of Cloud Connectors issues that are resolved in Cloudera Runtime 7.2.17.

CDPD-46175: HADOOP-18521. ABFS prefetching input stream corruption
CDPD-56046: S3 distcp --update overwrites date of target file, even when names and content match
Changed test to match the product. Distcp updates the date of the file to the current timestamp if a different file is moved there.
CDPD-50522: Data-con setup fails, causing some tests to run with invalid configs
Fixed in CDPQE-16900 /
CDPD-48449: distcp -update skips files of same size, name when transferring from Hdfs to S3
The Distcp -update option, may encounter potential inaccuracies by skipping the copy when doing incremental update of files with identical names and sizes during the transfer process from HDFS to S3 or ABFS. This occurs due to the absence of checksum verification between the files for different stores. In order to address this concern, we employ the modification time as a means to minimize the occurrence of incorrect skips. If the source file has been modified more recently than its corresponding destination file, we proceed with the copy operation; otherwise, the file is skipped.
CDPD-46543: HADOOP-18526. Leak of S3AInstrumentation instances via hadoop Metrics references
HADOOP-18526. Leak of S3AInstrumentation instances via hadoop Metrics references
CDPD-45959: Some tests fail with ssl3_get_server_certificate:certificate verify failed
"" has been set to "Default_JSSE". Switch to "Default" if the version of OpenSSL installed in your OS can successfully negotiate SSL connections with azure to achieve possibly improved performance.

Apache Patch Information

  • HADOOP-18526
  • HADOOP-18596
  • HADOOP-18521