Cloudera Docs

Flush options

The Azure Blob File System and OutputStream flush options are enabled by default. These flush options have an impact on performance. Those applications which call flush() frequently, for example, with every write of a line, might show poor performance. Hence, you must disable the flush option if your application does not need it.

  • Azure Blob File System Flush Options (fs.azure.enable.flush) - renders ABFS flush APIs - HFlush() and HSync(), to be no-op. Both the APIs ensure that data persists.
  • OutputStream Flush Options (fs.azure.disable.outputstream.flush) - renders OutputStream Flush() API to be a no-op in AbfsOutputStream. Hflush() being the only documented API that can provide persistent data transfer, flush() also attempting to persist buffered data will lead to performance issues.