org.apache.hadoop.hbase.io.hfile

Class CompoundBloomFilterWriter

java.lang.Object

org.apache.hadoop.hbase.io.hfile.CompoundBloomFilterBase

org.apache.hadoop.hbase.io.hfile.CompoundBloomFilterWriter

All Implemented Interfaces:

InlineBlockWriter, CellSink, ShipperListener, BloomFilterBase, BloomFilterWriter

@InterfaceAudience.Private
public class CompoundBloomFilterWriter
extends CompoundBloomFilterBase
implements BloomFilterWriter, InlineBlockWriter

Adds methods required for writing a compound Bloom filter to the data section of an HFile to the CompoundBloomFilter class.

Field Summary

Fields inherited from class org.apache.hadoop.hbase.io.hfile.CompoundBloomFilterBase

comparator, errorRate, hashType, numChunks, totalByteSize, totalKeyCount, totalMaxKeys, VERSION

Constructor Summary

Constructors

Constructor and Description
CompoundBloomFilterWriter(int chunkByteSizeHint, float errorRate, int hashType, int maxFold, boolean cacheOnWrite, CellComparator comparator, BloomType bloomType)

Method Summary

Modifier and Type	Method and Description
void	append(Cell cell) Append the given cell
void	beforeShipped() The action that needs to be performed before Shipper.shipped() is performed
void	blockWritten(long offset, int onDiskSize, int uncompressedSize) Called after a block has been written, and its offset, raw size, and compressed size have been determined.
void	compactBloom() Compact the Bloom filter before writing metadata & data to disk.
boolean	getCacheOnWrite()
org.apache.hadoop.io.Writable	getDataWriter() Get a writable interface into bloom filter data (the actual Bloom bits).
BlockType	getInlineBlockType() The type of blocks this block writer produces.
org.apache.hadoop.io.Writable	getMetaWriter() Get a writable interface into bloom filter meta data.
Cell	getPrevCell() Returns the previous cell written by this writer
boolean	shouldWriteBlock(boolean closing) Determines whether there is a new block to be written out.
void	writeInlineBlock(DataOutput out) Writes the block to the provided stream.

Methods inherited from class org.apache.hadoop.hbase.io.hfile.CompoundBloomFilterBase

getByteSize, getKeyCount, getMaxKeys

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.util.BloomFilterBase

getByteSize, getKeyCount, getMaxKeys

Constructor Detail

CompoundBloomFilterWriter

public CompoundBloomFilterWriter(int chunkByteSizeHint,
                                 float errorRate,
                                 int hashType,
                                 int maxFold,
                                 boolean cacheOnWrite,
                                 CellComparator comparator,
                                 BloomType bloomType)

Parameters:

chunkByteSizeHint - each chunk's size in bytes. The real chunk size might be different as required by the fold factor.

errorRate - target false positive rate

hashType - hash function type to use

maxFold - maximum degree of folding allowed

bloomType - the bloom type

Method Detail

shouldWriteBlock

public boolean shouldWriteBlock(boolean closing)

Description copied from interface: InlineBlockWriter

Determines whether there is a new block to be written out.

Specified by:

shouldWriteBlock in interface InlineBlockWriter

Parameters:

closing - whether the file is being closed, in which case we need to write out all available data and not wait to accumulate another block

append

public void append(Cell cell)
            throws IOException

Description copied from interface: CellSink

Append the given cell

Specified by:

append in interface CellSink

Parameters:

cell - the cell to be added

Throws:

IOException

beforeShipped

public void beforeShipped()
                   throws IOException

Description copied from interface: ShipperListener

The action that needs to be performed before Shipper.shipped() is performed

Specified by:

beforeShipped in interface ShipperListener

Throws:

IOException

getPrevCell

public Cell getPrevCell()

Description copied from interface: BloomFilterWriter

Returns the previous cell written by this writer

Specified by:

getPrevCell in interface BloomFilterWriter

Returns:

the previous cell

writeInlineBlock

public void writeInlineBlock(DataOutput out)
                      throws IOException

Description copied from interface: InlineBlockWriter

Writes the block to the provided stream. Must not write any magic records. Called only if InlineBlockWriter.shouldWriteBlock(boolean) returned true.

Specified by:

writeInlineBlock in interface InlineBlockWriter

Parameters:

out - a stream (usually a compressing stream) to write the block to

Throws:

IOException

blockWritten

public void blockWritten(long offset,
                         int onDiskSize,
                         int uncompressedSize)

Description copied from interface: InlineBlockWriter

Called after a block has been written, and its offset, raw size, and compressed size have been determined. Can be used to add an entry to a block index. If this type of inline blocks needs a block index, the inline block writer is responsible for maintaining it.

Specified by:

blockWritten in interface InlineBlockWriter

Parameters:

offset - the offset of the block in the stream

onDiskSize - the on-disk size of the block

uncompressedSize - the uncompressed size of the block

getInlineBlockType

public BlockType getInlineBlockType()

Description copied from interface: InlineBlockWriter

The type of blocks this block writer produces.

Specified by:

getInlineBlockType in interface InlineBlockWriter

compactBloom

public void compactBloom()

Description copied from interface: BloomFilterWriter

Compact the Bloom filter before writing metadata & data to disk.

Specified by:

compactBloom in interface BloomFilterWriter

getMetaWriter

public org.apache.hadoop.io.Writable getMetaWriter()

Description copied from interface: BloomFilterWriter

Get a writable interface into bloom filter meta data.

Specified by:

getMetaWriter in interface BloomFilterWriter

Returns:

a writable instance that can be later written to a stream

getDataWriter

public org.apache.hadoop.io.Writable getDataWriter()

Description copied from interface: BloomFilterWriter

Get a writable interface into bloom filter data (the actual Bloom bits). Not used for compound Bloom filters.

Specified by:

getDataWriter in interface BloomFilterWriter

Returns:

a writable instance that can be later written to a stream

getCacheOnWrite

public boolean getCacheOnWrite()

Specified by:

getCacheOnWrite in interface InlineBlockWriter

Returns:

true if inline blocks produced by this writer should be cached