org.apache.hadoop.hbase.wal

Class WALSplitter.OutputSink

java.lang.Object

org.apache.hadoop.hbase.wal.WALSplitter.OutputSink

Enclosing class:

WALSplitter

public abstract static class WALSplitter.OutputSink
extends Object

The following class is an abstraction class to provide a common interface to support both existing recovered edits file sink and region server WAL edits replay sink

Field Summary

Fields

Modifier and Type	Field and Description
protected Set<byte[]>	blacklistedRegions
protected boolean	closeAndCleanCompleted
protected WALSplitter.PipelineController	controller
protected WALSplitter.EntryBuffers	entryBuffers
protected int	numThreads
protected Map<byte[],Long>	regionMaximumEditLogSeqNum
protected CancelableProgressable	reporter
protected AtomicLong	skippedEdits
protected List<Path>	splits
protected Map<byte[],WALSplitter.SinkWriter>	writers
protected boolean	writersClosed
protected List<WALSplitter.WriterThread>	writerThreads

Constructor Summary

Constructors

Constructor and Description
OutputSink(WALSplitter.PipelineController controller, WALSplitter.EntryBuffers entryBuffers, int numWriters)

Method Summary

Modifier and Type	Method and Description
abstract void	append(WALSplitter.RegionEntryBuffer buffer)
protected boolean	finishWriting(boolean interrupt) Wait for writer threads to dump all info to the sink
abstract List<Path>	finishWritingAndClose()
boolean	flush() WriterThread call this function to help flush internal remaining edits in buffer before close
abstract int	getNumberOfRecoveredRegions()
abstract Map<byte[],Long>	getOutputCounts()
void	startWriterThreads() Start the threads that will pump data from the entryBuffers to the output files.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

controller

protected WALSplitter.PipelineController controller

entryBuffers

protected WALSplitter.EntryBuffers entryBuffers

writers

protected Map<byte[],WALSplitter.SinkWriter> writers

regionMaximumEditLogSeqNum

protected final Map<byte[],Long> regionMaximumEditLogSeqNum

writerThreads

protected final List<WALSplitter.WriterThread> writerThreads

blacklistedRegions

protected final Set<byte[]> blacklistedRegions

closeAndCleanCompleted

protected boolean closeAndCleanCompleted

writersClosed

protected boolean writersClosed

numThreads

protected final int numThreads

reporter

protected CancelableProgressable reporter

skippedEdits

protected AtomicLong skippedEdits

splits

protected List<Path> splits

Constructor Detail

OutputSink

public OutputSink(WALSplitter.PipelineController controller,
                  WALSplitter.EntryBuffers entryBuffers,
                  int numWriters)

Method Detail

startWriterThreads

public void startWriterThreads()

Start the threads that will pump data from the entryBuffers to the output files.

finishWriting

protected boolean finishWriting(boolean interrupt)
                         throws IOException

Wait for writer threads to dump all info to the sink

Returns:

true when there is no error

Throws:

IOException

finishWritingAndClose

public abstract List<Path> finishWritingAndClose()
                                          throws IOException

Throws:

IOException

getOutputCounts

public abstract Map<byte[],Long> getOutputCounts()

Returns:

a map from encoded region ID to the number of edits written out for that region.

getNumberOfRecoveredRegions

public abstract int getNumberOfRecoveredRegions()

Returns:

number of regions we've recovered

append

public abstract void append(WALSplitter.RegionEntryBuffer buffer)
                     throws IOException

Parameters:

buffer - A WAL Edit Entry

Throws:

IOException

flush

public boolean flush()
              throws IOException

WriterThread call this function to help flush internal remaining edits in buffer before close

Returns:

true when underlying sink has something to flush

Throws:

IOException