Data Durability Considerations

You can now read from Ozone data stored with Erasure Coding (EC) feature. EC can be considered as an alternative to replication.

Erasure Coding Overview
The Ozone EC feature provides data durability and fault-tolerance along with reduced storage space and ensures data durability similar to the Ratis THREE replication approach.
Verify the Related Query Option
Impala already supports the basic functionalities of erasure-coding. Before this release, EC was turned off by default because Impala lacked metrics, observability and DDL commands related to erasure coding.
Enable EC Replication
You can enable EC replication configuration cluster wide or at the bucket level or at file levels. The details of which can be found here.
Verify the EC Policies
You can verify the EC policies you set up on the File / Table / Partition level using the Impala commands.
Understand Query Performance
Querying an erasure coded data might have an impact on the performance. If you notice performance degradation, you can troubleshoot it by running the PROFILE command. You can also use CM to quickly identify the cause.
Monitor EC Metrics
Under Query Details in CM, you can check whether a query read erasure coded data via HDFS Erasure Coded Bytes Read.