org.apache.tephra.hbase.coprocessor

Class TransactionProcessor

java.lang.Object

org.apache.hadoop.hbase.coprocessor.BaseRegionObserver

org.apache.tephra.hbase.coprocessor.TransactionProcessor

All Implemented Interfaces:

org.apache.hadoop.hbase.Coprocessor, org.apache.hadoop.hbase.coprocessor.RegionObserver

public class TransactionProcessor
extends org.apache.hadoop.hbase.coprocessor.BaseRegionObserver

org.apache.hadoop.hbase.coprocessor.RegionObserver coprocessor that handles server-side processing for transactions:

• applies filtering to exclude data from invalid and in-progress transactions
• overrides the scanner returned for flush and compaction to drop data written by invalidated transactions, or expired due to TTL.

In order to use this coprocessor for transactions, configure the class on any table involved in transactions, or on all user tables by adding the following to hbase-site.xml: <property> <name>hbase.coprocessor.region.classes</name> <value>org.apache.tephra.hbase.coprocessor.TransactionProcessor</value> </property>

HBase Get and Scan operations should have the current transaction serialized on to the operation as an attribute: Transaction t = ...; Get get = new Get(...); TransactionCodec codec = new TransactionCodec(); codec.addToOperation(get, t);

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.coprocessor.RegionObserver

org.apache.hadoop.hbase.coprocessor.RegionObserver.MutationType

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.Coprocessor

org.apache.hadoop.hbase.Coprocessor.State

Field Summary

Fields

Modifier and Type	Field and Description
protected boolean	allowEmptyValues
protected Boolean	pruneEnable
protected boolean	readNonTxnData
protected Map<byte[],Long>	ttlByFamily
protected Long	txMaxLifetimeMillis

Fields inherited from interface org.apache.hadoop.hbase.Coprocessor

PRIORITY_HIGHEST, PRIORITY_LOWEST, PRIORITY_SYSTEM, PRIORITY_USER, VERSION

Constructor Summary

Constructors

Constructor and Description
TransactionProcessor()

Method Summary

Modifier and Type	Method and Description
protected org.apache.hadoop.hbase.regionserver.InternalScanner	createStoreScanner(org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment env, String action, TransactionVisibilityState snapshot, org.apache.hadoop.hbase.regionserver.Store store, List<? extends org.apache.hadoop.hbase.regionserver.KeyValueScanner> scanners, org.apache.hadoop.hbase.regionserver.ScanType type, long earliestPutTs)
protected void	ensureValidTxLifetime(org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment env, org.apache.hadoop.hbase.client.OperationWithAttributes op, Transaction tx) Make sure that the transaction is within the max valid transaction lifetime.
protected org.apache.hadoop.conf.Configuration	getConfiguration(org.apache.hadoop.hbase.CoprocessorEnvironment env) Fetch the Configuration that contains the properties required by the coprocessor.
protected org.apache.hadoop.hbase.filter.Filter	getTransactionFilter(Transaction tx, org.apache.hadoop.hbase.regionserver.ScanType type, org.apache.hadoop.hbase.filter.Filter filter) Derived classes can override this method to customize the filter used to return data visible for the current transaction.
protected CacheSupplier<TransactionStateCache>	getTransactionStateCacheSupplier(org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment env)
protected void	initializePruneState(org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment env) Refresh the properties related to transaction pruning.
void	postCompact(org.apache.hadoop.hbase.coprocessor.ObserverContext<org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment> e, org.apache.hadoop.hbase.regionserver.Store store, org.apache.hadoop.hbase.regionserver.StoreFile resultFile, org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest request)
void	postFlush(org.apache.hadoop.hbase.coprocessor.ObserverContext<org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment> e)
org.apache.hadoop.hbase.regionserver.InternalScanner	preCompactScannerOpen(org.apache.hadoop.hbase.coprocessor.ObserverContext<org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment> c, org.apache.hadoop.hbase.regionserver.Store store, List<? extends org.apache.hadoop.hbase.regionserver.KeyValueScanner> scanners, org.apache.hadoop.hbase.regionserver.ScanType scanType, long earliestPutTs, org.apache.hadoop.hbase.regionserver.InternalScanner s, org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest request)
void	preDelete(org.apache.hadoop.hbase.coprocessor.ObserverContext<org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment> e, org.apache.hadoop.hbase.client.Delete delete, org.apache.hadoop.hbase.regionserver.wal.WALEdit edit, org.apache.hadoop.hbase.client.Durability durability)
org.apache.hadoop.hbase.regionserver.InternalScanner	preFlushScannerOpen(org.apache.hadoop.hbase.coprocessor.ObserverContext<org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment> c, org.apache.hadoop.hbase.regionserver.Store store, org.apache.hadoop.hbase.regionserver.KeyValueScanner memstoreScanner, org.apache.hadoop.hbase.regionserver.InternalScanner scanner)
void	preGetOp(org.apache.hadoop.hbase.coprocessor.ObserverContext<org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment> e, org.apache.hadoop.hbase.client.Get get, List<org.apache.hadoop.hbase.Cell> results)
void	prePut(org.apache.hadoop.hbase.coprocessor.ObserverContext<org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment> e, org.apache.hadoop.hbase.client.Put put, org.apache.hadoop.hbase.regionserver.wal.WALEdit edit, org.apache.hadoop.hbase.client.Durability durability)
org.apache.hadoop.hbase.regionserver.RegionScanner	preScannerOpen(org.apache.hadoop.hbase.coprocessor.ObserverContext<org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment> e, org.apache.hadoop.hbase.client.Scan scan, org.apache.hadoop.hbase.regionserver.RegionScanner s)
protected void	resetPruneState() Stop and clear state related to pruning.
void	start(org.apache.hadoop.hbase.CoprocessorEnvironment e)
void	stop(org.apache.hadoop.hbase.CoprocessorEnvironment e)

Methods inherited from class org.apache.hadoop.hbase.coprocessor.BaseRegionObserver

postAppend, postBatchMutate, postBatchMutateIndispensably, postBulkLoadHFile, postCheckAndDelete, postCheckAndPut, postClose, postCloseRegionOperation, postCompact, postCompactSelection, postCompactSelection, postCompleteSplit, postDelete, postExists, postFlush, postGetClosestRowBefore, postGetOp, postIncrement, postIncrementColumnValue, postInstantiateDeleteTracker, postLogReplay, postMutationBeforeWAL, postOpen, postPut, postRollBackSplit, postScannerClose, postScannerFilterRow, postScannerNext, postScannerOpen, postSplit, postStartRegionOperation, postStoreFileReaderOpen, postWALRestore, postWALRestore, preAppend, preAppendAfterRowLock, preBatchMutate, preBulkLoadHFile, preCheckAndDelete, preCheckAndDeleteAfterRowLock, preCheckAndPut, preCheckAndPutAfterRowLock, preClose, preCompact, preCompact, preCompactScannerOpen, preCompactSelection, preCompactSelection, preExists, preFlush, preFlush, preGetClosestRowBefore, preIncrement, preIncrementAfterRowLock, preIncrementColumnValue, preOpen, prePrepareTimeStampForDeleteVersion, preRollBackSplit, preScannerClose, preScannerNext, preSplit, preSplit, preSplitAfterPONR, preSplitBeforePONR, preStoreFileReaderOpen, preStoreScannerOpen, preWALRestore, preWALRestore

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

pruneEnable

protected volatile Boolean pruneEnable

txMaxLifetimeMillis

protected volatile Long txMaxLifetimeMillis

ttlByFamily

protected Map<byte[],Long> ttlByFamily

allowEmptyValues

protected boolean allowEmptyValues

readNonTxnData

protected boolean readNonTxnData

Constructor Detail

TransactionProcessor

public TransactionProcessor()

Method Detail

start

public void start(org.apache.hadoop.hbase.CoprocessorEnvironment e)
           throws IOException

Specified by:

start in interface org.apache.hadoop.hbase.Coprocessor

Overrides:

start in class org.apache.hadoop.hbase.coprocessor.BaseRegionObserver

Throws:

IOException

getConfiguration

@Nullable
protected org.apache.hadoop.conf.Configuration getConfiguration(org.apache.hadoop.hbase.CoprocessorEnvironment env)

Fetch the Configuration that contains the properties required by the coprocessor. By default, the HBase configuration is returned. This method will never return null in Tephra but the derived classes might do so if Configuration is not available temporarily (for example, if it is being fetched from a HBase Table.

Parameters:

env - RegionCoprocessorEnvironment of the Region to which the coprocessor is associated

Returns:

Configuration, can be null if it is not available

getTransactionStateCacheSupplier

protected CacheSupplier<TransactionStateCache> getTransactionStateCacheSupplier(org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment env)

stop

public void stop(org.apache.hadoop.hbase.CoprocessorEnvironment e)
          throws IOException

Specified by:

stop in interface org.apache.hadoop.hbase.Coprocessor

Overrides:

stop in class org.apache.hadoop.hbase.coprocessor.BaseRegionObserver

Throws:

IOException

preGetOp

public void preGetOp(org.apache.hadoop.hbase.coprocessor.ObserverContext<org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment> e,
                     org.apache.hadoop.hbase.client.Get get,
                     List<org.apache.hadoop.hbase.Cell> results)
              throws IOException

Specified by:

preGetOp in interface org.apache.hadoop.hbase.coprocessor.RegionObserver

Overrides:

preGetOp in class org.apache.hadoop.hbase.coprocessor.BaseRegionObserver

Throws:

IOException

prePut

public void prePut(org.apache.hadoop.hbase.coprocessor.ObserverContext<org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment> e,
                   org.apache.hadoop.hbase.client.Put put,
                   org.apache.hadoop.hbase.regionserver.wal.WALEdit edit,
                   org.apache.hadoop.hbase.client.Durability durability)
            throws IOException

Specified by:

prePut in interface org.apache.hadoop.hbase.coprocessor.RegionObserver

Overrides:

prePut in class org.apache.hadoop.hbase.coprocessor.BaseRegionObserver

Throws:

IOException

preDelete

public void preDelete(org.apache.hadoop.hbase.coprocessor.ObserverContext<org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment> e,
                      org.apache.hadoop.hbase.client.Delete delete,
                      org.apache.hadoop.hbase.regionserver.wal.WALEdit edit,
                      org.apache.hadoop.hbase.client.Durability durability)
               throws IOException

Specified by:

preDelete in interface org.apache.hadoop.hbase.coprocessor.RegionObserver

Overrides:

preDelete in class org.apache.hadoop.hbase.coprocessor.BaseRegionObserver

Throws:

IOException

preScannerOpen

public org.apache.hadoop.hbase.regionserver.RegionScanner preScannerOpen(org.apache.hadoop.hbase.coprocessor.ObserverContext<org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment> e,
                                                                         org.apache.hadoop.hbase.client.Scan scan,
                                                                         org.apache.hadoop.hbase.regionserver.RegionScanner s)
                                                                  throws IOException

Specified by:

preScannerOpen in interface org.apache.hadoop.hbase.coprocessor.RegionObserver

Overrides:

preScannerOpen in class org.apache.hadoop.hbase.coprocessor.BaseRegionObserver

Throws:

IOException

preFlushScannerOpen

public org.apache.hadoop.hbase.regionserver.InternalScanner preFlushScannerOpen(org.apache.hadoop.hbase.coprocessor.ObserverContext<org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment> c,
                                                                                org.apache.hadoop.hbase.regionserver.Store store,
                                                                                org.apache.hadoop.hbase.regionserver.KeyValueScanner memstoreScanner,
                                                                                org.apache.hadoop.hbase.regionserver.InternalScanner scanner)
                                                                         throws IOException

Specified by:

preFlushScannerOpen in interface org.apache.hadoop.hbase.coprocessor.RegionObserver

Overrides:

preFlushScannerOpen in class org.apache.hadoop.hbase.coprocessor.BaseRegionObserver

Throws:

IOException

postFlush

public void postFlush(org.apache.hadoop.hbase.coprocessor.ObserverContext<org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment> e)
               throws IOException

Specified by:

postFlush in interface org.apache.hadoop.hbase.coprocessor.RegionObserver

Overrides:

postFlush in class org.apache.hadoop.hbase.coprocessor.BaseRegionObserver

Throws:

IOException

preCompactScannerOpen

public org.apache.hadoop.hbase.regionserver.InternalScanner preCompactScannerOpen(org.apache.hadoop.hbase.coprocessor.ObserverContext<org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment> c,
                                                                                  org.apache.hadoop.hbase.regionserver.Store store,
                                                                                  List<? extends org.apache.hadoop.hbase.regionserver.KeyValueScanner> scanners,
                                                                                  org.apache.hadoop.hbase.regionserver.ScanType scanType,
                                                                                  long earliestPutTs,
                                                                                  org.apache.hadoop.hbase.regionserver.InternalScanner s,
                                                                                  org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest request)
                                                                           throws IOException

Specified by:

preCompactScannerOpen in interface org.apache.hadoop.hbase.coprocessor.RegionObserver

Overrides:

preCompactScannerOpen in class org.apache.hadoop.hbase.coprocessor.BaseRegionObserver

Throws:

IOException

postCompact

public void postCompact(org.apache.hadoop.hbase.coprocessor.ObserverContext<org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment> e,
                        org.apache.hadoop.hbase.regionserver.Store store,
                        org.apache.hadoop.hbase.regionserver.StoreFile resultFile,
                        org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest request)
                 throws IOException

Specified by:

postCompact in interface org.apache.hadoop.hbase.coprocessor.RegionObserver

Overrides:

postCompact in class org.apache.hadoop.hbase.coprocessor.BaseRegionObserver

Throws:

IOException

createStoreScanner

protected org.apache.hadoop.hbase.regionserver.InternalScanner createStoreScanner(org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment env,
                                                                                  String action,
                                                                                  TransactionVisibilityState snapshot,
                                                                                  org.apache.hadoop.hbase.regionserver.Store store,
                                                                                  List<? extends org.apache.hadoop.hbase.regionserver.KeyValueScanner> scanners,
                                                                                  org.apache.hadoop.hbase.regionserver.ScanType type,
                                                                                  long earliestPutTs)
                                                                           throws IOException

Throws:

IOException

ensureValidTxLifetime

protected void ensureValidTxLifetime(org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment env,
                                     org.apache.hadoop.hbase.client.OperationWithAttributes op,
                                     @Nullable
                                     Transaction tx)
                              throws IOException

Make sure that the transaction is within the max valid transaction lifetime.

Parameters:

env - RegionCoprocessorEnvironment of the Region to which the coprocessor is associated

op - OperationWithAttributes HBase operation to access its attributes if required

tx - Transaction supplied by the

Throws:

org.apache.hadoop.hbase.DoNotRetryIOException - thrown if the transaction is older than the max lifetime of a transaction IOException throw if the value of max lifetime of transaction is unavailable

IOException

getTransactionFilter

protected org.apache.hadoop.hbase.filter.Filter getTransactionFilter(Transaction tx,
                                                                     org.apache.hadoop.hbase.regionserver.ScanType type,
                                                                     org.apache.hadoop.hbase.filter.Filter filter)

Derived classes can override this method to customize the filter used to return data visible for the current transaction.

Parameters:

tx - the current transaction to apply

type - the type of scan operation being performed

initializePruneState

protected void initializePruneState(org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment env)

Refresh the properties related to transaction pruning. This method needs to be invoked if there is change in the prune related properties after clearing the state by calling resetPruneState().

Parameters:

env - RegionCoprocessorEnvironment of this region

resetPruneState

protected void resetPruneState()

Stop and clear state related to pruning.