Data Durability Considerations

When you store the data in Ozone, this release provides an Erasure Coding (EC) feature. EC can be considered as an alternative to replication.

Erasure Coding Overview
The Ozone EC feature provides data durability and fault-tolerance along with reduced storage space and ensures data durability similar to the Ratis THREE replication approach.
Verify the Related Query Option
Impala already supports the basic functionalities of erasure-coding. Before this release, EC was turned off by default because Impala lacked metrics, observability and DDL commands related to erasure coding.
Enable EC Replication
You can enable EC replication configuration cluster wide or at the bucket level or at file levels. The details of which can be found here.
Verify the EC Policies
You can verify the EC policies you set up on the File / Table / Partition level using the Impala commands.
Understanding Query Performance
Querying an erasure coded data might have an impact on the performance. If you notice performance degradation, you can troubleshoot it by running the PROFILE command. The PROFILE statement gives you an outline of the logical steps that a query will perform. It will include erasure coding related information in the profile.