Filter types
HBase includes several filter types, as well as the ability to group filters together and create your own custom filters.
- KeyOnlyFilter - takes no arguments. Returns the key portion of each key-value
pair.
Syntax: KeyOnlyFilter ()
- FirstKeyOnlyFilter - takes no arguments. Returns the key portion of the first
key-value pair.
Syntax: FirstKeyOnlyFilter ()
- PrefixFilter - takes a single argument, a prefix of a row key. It returns only
those key-values present in a row that start with the specified row
prefix
Syntax: PrefixFilter (‘<row_prefix>’) Example: PrefixFilter (‘Row’)
- ColumnPrefixFilter - takes a single argument, a column prefix. It returns only
those key-values present in a column that starts with the specified column
prefix.
Syntax: ColumnPrefixFilter (‘<column_prefix>’) Example: ColumnPrefixFilter (‘Col’)
- MultipleColumnPrefixFilter - takes a list of column prefixes. It returns
key-values that are present in a column that starts with any of the specified
column
prefixes.
Syntax: MultipleColumnPrefixFilter (‘<column_prefix>’, ‘<column_prefix>’, …, ‘<column_prefix>’) Example: MultipleColumnPrefixFilter (‘Col1’, ‘Col2’)
- ColumnCountGetFilter - takes one argument, a limit. It returns the first
limit
number of columns in the table.Syntax: ColumnCountGetFilter (‘<limit>’) Example: ColumnCountGetFilter (4)
- PageFilter - takes one argument, a page size. It returns
page size
number of rows from the table.Syntax: PageFilter (‘<page_size>’) Example: PageFilter (2)
- ColumnPaginationFilter - takes two arguments, a limit and offset. It returns
limit number of columns after offset number of columns. It does this for all the
rows.
Syntax: ColumnPaginationFilter (‘<limit>’, ‘<offset>’) Example: ColumnPaginationFilter (3, 5)
- InclusiveStopFilter - takes one argument, a row key on which to stop scanning. It
returns all key-values present in rows up to and including the specified
row.
Syntax: InclusiveStopFilter (‘<stop_row_key>’) Example: InclusiveStopFilter (‘Row2’)
- TimeStampsFilter - takes a list of timestamps. It returns those key-values whose
timestamps matches any of the specified
timestamps.
Syntax: TimeStampsFilter (<timestamp>, <timestamp>, ... ,<timestamp>) Example: TimeStampsFilter (5985489, 48895495, 58489845945)
- RowFilter - takes a compare operator and a comparator. It compares each row key
with the comparator using the compare operator and if the comparison returns
true
, it returns all the key-values in that row.Syntax: RowFilter (<compareOp>, ‘<row_comparator>’) Example: RowFilter (<=, ‘binary:xyz)
- FamilyFilter - takes a compare operator and a comparator. It compares each family
name with the comparator using the compare operator and if the comparison returns
true
, it returns all the key-values in that family.Syntax: FamilyFilter (<compareOp>, ‘<family_comparator>’) Example: FamilyFilter (>=, ‘binaryprefix:FamilyB’)
- QualifierFilter - takes a compare operator and a comparator. It compares each
qualifier name with the comparator using the compare operator and if the comparison
returns
true
, it returns all the key-values in that column.Syntax: QualifierFilter (<compareOp>, ‘<qualifier_comparator>’) Example: QualifierFilter (=, ‘substring:Column1’)
- ValueFilter - takes a compare operator and a comparator. It compares each value
with the comparator using the compare operator and if the comparison returns
true
, it returns that key-value.Syntax: ValueFilter (<compareOp>, ‘<value_comparator>’) Example: ValueFilter (!=, ‘binary:Value’)
- DependentColumnFilter - takes two arguments required arguments, a family and a
qualifier. It tries to locate this column in each row and returns all key-values in that
row that have the same timestamp. If the row does not contain the specified column, none
of the key-values in that row will be returned.
The filter can also take an optional boolean argument,
dropDependentColumn
. If set totrue
, the column used for the filter does not get returned.The filter can also take two more additional optional arguments, a compare operator and a value comparator, which are further checks in addition to the family and qualifier. If the dependent column is found, its value should also pass the value check. If it does pass the value check, only then is its timestamp taken into consideration.
Syntax: DependentColumnFilter (‘<family>’, ‘<qualifier>’, <boolean>, <compare operator>, ‘<value comparator’) DependentColumnFilter (‘<family>’, ‘<qualifier>’, <boolean>) DependentColumnFilter (‘<family>’, ‘<qualifier>’) Example: DependentColumnFilter (‘conf’, ‘blacklist’, false, >=, ‘zebra’) DependentColumnFilter (‘conf’, ‘blacklist’, true) DependentColumnFilter (‘conf’, ‘blacklist’)
- SingleColumnValueFilter - takes a column family, a qualifier, a compare operator
and a comparator. If the specified column is not found, all the columns of that row will
be emitted. If the column is found and the comparison with the comparator returns
true
, all the columns of the row will be emitted. If the condition fails, the row will not be emitted.This filter also takes two additional optional boolean arguments,
filterIfColumnMissing
andsetLatestVersionOnly
.If the
filterIfColumnMissing
flag is set totrue
, the columns of the row will not be emitted if the specified column to check is not found in the row. The default value isfalse
.If the
setLatestVersionOnly
flag is set tofalse
, it will test previous versions (timestamps) in addition to the most recent. The default value istrue
.These flags are optional and dependent on each other. You must set neither or both of them together.
Syntax: SingleColumnValueFilter (‘<family>’, ‘<qualifier>’, <compare operator>, ‘<comparator>’, <filterIfColumnMissing_boolean>, <latest_version_boolean>) Syntax: SingleColumnValueFilter (‘<family>’, ‘<qualifier>’, <compare operator>, ‘<comparator>’) Example: SingleColumnValueFilter (‘FamilyA’, ‘Column1’, <=, ‘abc’, true, false) Example: SingleColumnValueFilter ('FamilyA’, ‘Column1’, <=, ‘abc’)
- SingleColumnValueExcludeFilter - takes the same arguments and behaves same as
SingleColumnValueFilter
. However, if the column is found and the condition passes, all the columns of the row will be emitted except for the tested column value.Syntax: SingleColumnValueExcludeFilter (<family>, <qualifier>, <compare operators>, <comparator>, <latest_version_boolean>, <filterIfColumnMissing_boolean>) Syntax: SingleColumnValueExcludeFilter (<family>, <qualifier>, <compare operator> <comparator>) Example: SingleColumnValueExcludeFilter (‘FamilyA’, ‘Column1’, ‘<=’, ‘abc’, ‘false’, ‘true’) Example: SingleColumnValueExcludeFilter (‘FamilyA’, ‘Column1’, ‘<=’, ‘abc’)
- ColumnRangeFilter - takes either
minColumn
,maxColumn
, or both. Returns only those keys with columns that are betweenminColumn
andmaxColumn
. It also takes two boolean variables to indicate whether to include theminColumn
andmaxColumn
or not. If you don’t want to set theminColumn
or themaxColumn
, you can pass in an empty argument.Syntax: ColumnRangeFilter (‘<minColumn >’, <minColumnInclusive_bool>, ‘<maxColumn>’, <maxColumnInclusive_bool>) Example: ColumnRangeFilter (‘abc’, true, ‘xyz’, false)
- Custom Filter - You can create a custom filter by implementing the Filter class. The JAR must be available on all RegionServers.