Cloud Data Access
Also available as:
loading table of contents...

Limitations When Using DistCp with S3

When using DistCp with data in S3, consider the following limitations:

  • The -append option is not supported.

  • The -diff option is not supported.

  • The -atomic option causes a rename of the temporary data, so significantly increases the time to commit work at the end of the operation. Furthermore, as S3A does not offer atomic renames of directories, the -atomic operation doesn't actually deliver what is promised. Avoid using this option.

  • All -p options, including those to preserve permissions, user and group information, attributes checksums, and replication are ignored.

  • CRC checking will not be performed, irrespective of the value of the -skipCrc flag.