Package org.apache.cassandra.cdc.scanner

Class CdcScannerBuilder

  • public class CdcScannerBuilder
extends java.lang.Object
    The core CDC logic used to execute a single microbatch that reads a set of SSTables, de-duplicates mutations across replicas and builds into a `org.apache.cassandra.spark.reader.StreamScanner` for consumption.

    • Field Summary

      Fields 
      Modifier and Type Field Description
      protected org.apache.cassandra.cdc.api.CassandraSource cassandraSource  
      protected org.apache.cassandra.bridge.CdcBridge cdcBridge  
      protected org.apache.cassandra.cdc.api.CdcOptions cdcOptions  
      protected int partitionId  

    • Constructor Summary

      Constructors 
      Constructor Description
      CdcScannerBuilder​(org.apache.cassandra.bridge.CdcBridge cdcBridge, int partitionId, org.apache.cassandra.cdc.api.CdcOptions cdcOptions, org.apache.cassandra.cdc.stats.ICdcStats stats, org.apache.cassandra.bridge.TokenRange tokenRange, org.apache.cassandra.cdc.state.CdcState startState, org.apache.cassandra.spark.utils.AsyncExecutor executor, boolean readCommitLogHeader, java.util.Map<org.apache.cassandra.spark.data.partitioner.CassandraInstance,​java.util.List<org.apache.cassandra.cdc.api.CommitLog>> logs, org.apache.cassandra.cdc.api.CassandraSource cassandraSource)  

    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      org.apache.cassandra.cdc.scanner.CdcStreamScanner build()  
      org.apache.cassandra.cdc.scanner.CdcStreamScanner buildStreamScanner​(java.util.Collection<org.apache.cassandra.db.commitlog.PartitionUpdateWrapper> updates, org.apache.cassandra.cdc.state.CdcState endState)
      Return a CdcSortedStreamScanner to iterate over a collection of Cdc updates
      static java.util.stream.Stream<org.apache.cassandra.cdc.api.CommitLog> sortAndLimit​(int partitionId, java.util.Collection<org.apache.cassandra.cdc.api.CommitLog> logs, int maxCommitLogsPerInstance, org.apache.cassandra.cdc.api.CommitLogMarkers markers, org.apache.cassandra.cdc.stats.ICdcStats stats)  

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Field Detail

      • cdcBridge

        protected final org.apache.cassandra.bridge.CdcBridge cdcBridge

      • cassandraSource

        protected final org.apache.cassandra.cdc.api.CassandraSource cassandraSource

      • cdcOptions

        protected final org.apache.cassandra.cdc.api.CdcOptions cdcOptions

      • partitionId

        protected final int partitionId

    • Constructor Detail

      • CdcScannerBuilder

        public CdcScannerBuilder​(org.apache.cassandra.bridge.CdcBridge cdcBridge,
                         int partitionId,
                         org.apache.cassandra.cdc.api.CdcOptions cdcOptions,
                         org.apache.cassandra.cdc.stats.ICdcStats stats,
                         @Nullable
                         org.apache.cassandra.bridge.TokenRange tokenRange,
                         @NotNull
                         org.apache.cassandra.cdc.state.CdcState startState,
                         @NotNull
                         org.apache.cassandra.spark.utils.AsyncExecutor executor,
                         boolean readCommitLogHeader,
                         @NotNull
                         java.util.Map<org.apache.cassandra.spark.data.partitioner.CassandraInstance,​java.util.List<org.apache.cassandra.cdc.api.CommitLog>> logs,
                         org.apache.cassandra.cdc.api.CassandraSource cassandraSource)

    • Method Detail

      • sortAndLimit

        public static java.util.stream.Stream<org.apache.cassandra.cdc.api.CommitLog> sortAndLimit​(int partitionId,
                                                                                           java.util.Collection<org.apache.cassandra.cdc.api.CommitLog> logs,
                                                                                           int maxCommitLogsPerInstance,
                                                                                           @NotNull
                                                                                           org.apache.cassandra.cdc.api.CommitLogMarkers markers,
                                                                                           org.apache.cassandra.cdc.stats.ICdcStats stats)

      • build

        public org.apache.cassandra.cdc.scanner.CdcStreamScanner build()

      • buildStreamScanner

        public org.apache.cassandra.cdc.scanner.CdcStreamScanner buildStreamScanner​(java.util.Collection<org.apache.cassandra.db.commitlog.PartitionUpdateWrapper> updates,
                                                                            @NotNull
                                                                            org.apache.cassandra.cdc.state.CdcState endState)
        Return a CdcSortedStreamScanner to iterate over a collection of Cdc updates
        Parameters:
        updates - collection of cdc updates read from the commit logs in this micro-batch
        endState - resulting the cdc state to be persisted once this micro-batch completes successfully.
        Returns:
        a CdcSortedStreamScanner