Performance and scalability limitations to consider for replication policies

Before you create a replication policy, you must consider a few performance and scalability limitations.

The following list contains the performance and scalability limitations that you must consider for replication policies:
  • The maximum number of files for a single replication job is 100 million.
  • The maximum number of files for a replication policy that runs more frequently than once in 8 hours is 10 million.
  • The throughput of the replication job depends on the absolute read and write throughput of the source and destination clusters.
  • Regular rebalancing of your HDFS clusters is required for efficient operation of replications.