Package org.apache.cassandra.spark.sparksql

Class SparkRowIterator

  • All Implemented Interfaces:
    java.io.Closeable, java.lang.AutoCloseable, org.apache.spark.sql.connector.read.PartitionReader<org.apache.spark.sql.catalyst.InternalRow>
    public class SparkRowIterator
extends org.apache.cassandra.spark.sparksql.RowIterator<T>
implements org.apache.spark.sql.connector.read.PartitionReader<org.apache.spark.sql.catalyst.InternalRow>
    Wrapper iterator around SparkCellIterator to normalize cells into Spark SQL rows

    • Field Summary

      • Fields inherited from class org.apache.cassandra.spark.sparksql.RowIterator

        it, requiredColumns, row

    • Constructor Summary

      Constructors 
      Constructor Description
      SparkRowIterator​(int partitionId, DataLayer dataLayer)  
      SparkRowIterator​(int partitionId, DataLayer dataLayer, org.apache.spark.sql.types.StructType requiredSchema, java.util.List<org.apache.cassandra.spark.sparksql.filters.PartitionKeyFilter> partitionKeyFilters)  

    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      protected static org.apache.cassandra.spark.sparksql.CellIterator buildCellIterator​(int partitionId, org.apache.cassandra.spark.data.CqlTable cqlTable, org.apache.spark.sql.types.StructType requiredSchema, DataLayer dataLayer, java.util.List<org.apache.cassandra.spark.sparksql.filters.PartitionKeyFilter> partitionKeyFilters)  
      protected static org.apache.cassandra.spark.sparksql.RowBuilder<org.apache.spark.sql.catalyst.expressions.GenericInternalRow> decorate​(org.apache.spark.sql.types.StructType requiredSchema, org.apache.cassandra.spark.sparksql.RowBuilder<org.apache.spark.sql.catalyst.expressions.GenericInternalRow> builder, java.util.List<org.apache.cassandra.spark.config.SchemaFeature> features)  
      org.apache.cassandra.spark.sparksql.FullRowBuilder<T> newFullRowBuilder()  
      org.apache.cassandra.spark.sparksql.PartialRowBuilder<T> newPartialBuilder()  
      org.apache.spark.sql.catalyst.expressions.GenericInternalRow rowBuilder​(java.lang.Object[] valueArray)
      Maps the Object[] valueArray generated by the RowIterator to the expected type

      • Methods inherited from class org.apache.cassandra.spark.sparksql.RowIterator

        close, get, newBuilder, next, rowMapIterator

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

      • Methods inherited from interface java.io.Closeable

        close

      • Methods inherited from interface org.apache.spark.sql.connector.read.PartitionReader

        currentMetricsValues, get, next

    • Constructor Detail

      • SparkRowIterator

        public SparkRowIterator​(int partitionId,
                        @NotNull
                        DataLayer dataLayer)

      • SparkRowIterator

        public SparkRowIterator​(int partitionId,
                        @NotNull
                        DataLayer dataLayer,
                        @Nullable
                        org.apache.spark.sql.types.StructType requiredSchema,
                        @NotNull
                        java.util.List<org.apache.cassandra.spark.sparksql.filters.PartitionKeyFilter> partitionKeyFilters)

    • Method Detail

      • decorate

        protected static org.apache.cassandra.spark.sparksql.RowBuilder<org.apache.spark.sql.catalyst.expressions.GenericInternalRow> decorate​(@Nullable
                                                                                                                                       org.apache.spark.sql.types.StructType requiredSchema,
                                                                                                                                       org.apache.cassandra.spark.sparksql.RowBuilder<org.apache.spark.sql.catalyst.expressions.GenericInternalRow> builder,
                                                                                                                                       java.util.List<org.apache.cassandra.spark.config.SchemaFeature> features)

      • rowBuilder

        public org.apache.spark.sql.catalyst.expressions.GenericInternalRow rowBuilder​(java.lang.Object[] valueArray)
        Maps the Object[] valueArray generated by the RowIterator to the expected type
        Returns:
        a value of type `T`

      • buildCellIterator

        protected static org.apache.cassandra.spark.sparksql.CellIterator buildCellIterator​(int partitionId,
                                                                                    org.apache.cassandra.spark.data.CqlTable cqlTable,
                                                                                    @Nullable
                                                                                    org.apache.spark.sql.types.StructType requiredSchema,
                                                                                    @NotNull
                                                                                    DataLayer dataLayer,
                                                                                    @NotNull
                                                                                    java.util.List<org.apache.cassandra.spark.sparksql.filters.PartitionKeyFilter> partitionKeyFilters)

      • newPartialBuilder

        public org.apache.cassandra.spark.sparksql.PartialRowBuilder<T> newPartialBuilder()
        Specified by:
        newPartialBuilder in class org.apache.cassandra.spark.sparksql.RowIterator<T>

      • newFullRowBuilder

        public org.apache.cassandra.spark.sparksql.FullRowBuilder<T> newFullRowBuilder()
        Specified by:
        newFullRowBuilder in class org.apache.cassandra.spark.sparksql.RowIterator<T>