org.apache.hadoop.hbase.spark

Class SparkSQLPushDownFilter

java.lang.Object

org.apache.hadoop.hbase.filter.Filter

org.apache.hadoop.hbase.filter.FilterBase

org.apache.hadoop.hbase.spark.SparkSQLPushDownFilter

@InterfaceAudience.Private
public class SparkSQLPushDownFilter
extends FilterBase

This filter will push down all qualifier logic given to us by SparkSQL so that we have make the filters at the region server level and avoid sending the data back to the client to be filtered.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.filter.Filter

Filter.ReturnCode

Field Summary

Fields

Modifier and Type	Field and Description
protected static org.slf4j.Logger	log

Fields inherited from class org.apache.hadoop.hbase.filter.Filter

reversed

Constructor Summary

Constructors

Constructor and Description
SparkSQLPushDownFilter(org.apache.hadoop.hbase.spark.DynamicLogicExpression dynamicLogicExpression, byte[][] valueFromQueryArray, HashMap<org.apache.hadoop.hbase.spark.ByteArrayComparable,HashMap<org.apache.hadoop.hbase.spark.ByteArrayComparable,String>> currentCellToColumnIndexMap, String encoderClassName)
SparkSQLPushDownFilter(org.apache.hadoop.hbase.spark.DynamicLogicExpression dynamicLogicExpression, byte[][] valueFromQueryArray, scala.collection.mutable.MutableList<org.apache.spark.sql.datasources.hbase.Field> fields, String encoderClassName)

Method Summary

Modifier and Type	Method and Description
Filter.ReturnCode	filterCell(Cell c) A way to filter based on the column family, column qualifier and/or the column value.
boolean	filterRow() Filters that never filter by rows based on previously gathered state from Filter.filterCell(Cell) can inherit this implementation that never filters a row.
static SparkSQLPushDownFilter	parseFrom(byte[] pbBytes)
byte[]	toByteArray() Return length 0 byte array for Filters that don't require special serialization

Methods inherited from class org.apache.hadoop.hbase.filter.FilterBase

createFilterFromArguments, filterAllRemaining, filterRowCells, filterRowKey, filterRowKey, getNextCellHint, hasFilterRow, isFamilyEssential, reset, toString, transformCell

Methods inherited from class org.apache.hadoop.hbase.filter.Filter

filterKeyValue, isReversed, setReversed

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

log

protected static final org.slf4j.Logger log

Constructor Detail

SparkSQLPushDownFilter

public SparkSQLPushDownFilter(org.apache.hadoop.hbase.spark.DynamicLogicExpression dynamicLogicExpression,
                              byte[][] valueFromQueryArray,
                              HashMap<org.apache.hadoop.hbase.spark.ByteArrayComparable,HashMap<org.apache.hadoop.hbase.spark.ByteArrayComparable,String>> currentCellToColumnIndexMap,
                              String encoderClassName)

SparkSQLPushDownFilter

public SparkSQLPushDownFilter(org.apache.hadoop.hbase.spark.DynamicLogicExpression dynamicLogicExpression,
                              byte[][] valueFromQueryArray,
                              scala.collection.mutable.MutableList<org.apache.spark.sql.datasources.hbase.Field> fields,
                              String encoderClassName)

Method Detail

filterCell

public Filter.ReturnCode filterCell(Cell c)
                             throws IOException

Description copied from class: Filter

A way to filter based on the column family, column qualifier and/or the column value. Return code is described below. This allows filters to filter only certain number of columns, then terminate without matching ever column. If filterRowKey returns true, filterCell needs to be consistent with it. filterCell can assume that filterRowKey has already been called for the row. If your filter returns ReturnCode.NEXT_ROW, it should return ReturnCode.NEXT_ROW until Filter.reset() is called just in case the caller calls for the next row. Concrete implementers can signal a failure condition in their code by throwing an IOException.

Overrides:

filterCell in class Filter

Parameters:

c - the Cell in question

Returns:

code as described below

Throws:

IOException - in case an I/O or an filter specific failure needs to be signaled.

See Also:

Filter.ReturnCode

filterRow

public boolean filterRow()
                  throws IOException

Description copied from class: FilterBase

Filters that never filter by rows based on previously gathered state from Filter.filterCell(Cell) can inherit this implementation that never filters a row. Last chance to veto row based on previous Filter.filterCell(Cell) calls. The filter needs to retain state then return a particular value for this call if they wish to exclude a row if a certain column is missing (for example). Concrete implementers can signal a failure condition in their code by throwing an IOException.

Overrides:

filterRow in class FilterBase

Returns:

true to exclude row, false to include row.

Throws:

IOException - in case an I/O or an filter specific failure needs to be signaled.

parseFrom

public static SparkSQLPushDownFilter parseFrom(byte[] pbBytes)
                                        throws DeserializationException

Parameters:

pbBytes - A pb serialized instance

Returns:

An instance of SparkSQLPushDownFilter

Throws:

DeserializationException

toByteArray

public byte[] toByteArray()

Description copied from class: FilterBase

Return length 0 byte array for Filters that don't require special serialization

Overrides:

toByteArray in class FilterBase

Returns:

The filter serialized using pb