Package org.apache.cassandra.spark.bulkwriter.cloudstorage

Class CloudStorageDataTransferApiImpl

java.lang.Object

org.apache.cassandra.spark.bulkwriter.cloudstorage.CloudStorageDataTransferApiImpl

All Implemented Interfaces:

CloudStorageDataTransferApi

public class CloudStorageDataTransferApiImpl
extends java.lang.Object
implements CloudStorageDataTransferApi

Encapsulates transfer APIs used by CloudStorageStreamSession. StorageClient is used to interact with S3 and upload SSTables bundles to S3 bucket. It also has SidecarClient to call relevant sidecar APIs.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	CloudStorageDataTransferApiImpl.ExecutorCreateSliceRetryPolicy	SidecarClient by default retries till 200 Http response.

Constructor Summary

Constructors

Constructor	Description
CloudStorageDataTransferApiImpl(JobInfo jobInfo, o.a.c.sidecar.client.shaded.client.SidecarClient sidecarClient, StorageClient storageClient, java.lang.String clusterId)

Method Summary

Modifier and Type	Method	Description
void	abortRestoreJob()
void	createRestoreJob(o.a.c.sidecar.client.shaded.common.request.data.CreateRestoreJobRequestPayload createRestoreJobRequestPayload)
java.util.concurrent.CompletableFuture<java.lang.Void>	createRestoreSliceFromDriver(o.a.c.sidecar.client.shaded.client.SidecarInstance sidecarInstance, o.a.c.sidecar.client.shaded.common.request.data.CreateSliceRequestPayload createSliceRequestPayload)	Called from driver level to create a restore slice asynchronously.
void	createRestoreSliceFromExecutor(o.a.c.sidecar.client.shaded.client.SidecarInstance sidecarInstance, o.a.c.sidecar.client.shaded.common.request.data.CreateSliceRequestPayload createSliceRequestPayload)	Called from task level to create a restore slice.
JobInfo	jobInfo()
o.a.c.sidecar.client.shaded.common.response.data.RestoreJobSummaryResponsePayload	restoreJobSummary()
o.a.c.sidecar.client.shaded.client.SidecarClient	sidecarClient()
void	updateRestoreJob(o.a.c.sidecar.client.shaded.common.request.data.UpdateRestoreJobRequestPayload updateRestoreJobRequestPayload)
BundleStorageObject	uploadBundle(StorageCredentials writeCredentials, Bundle bundle)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.cassandra.spark.bulkwriter.cloudstorage.CloudStorageDataTransferApi

handleInterruption

Constructor Detail

CloudStorageDataTransferApiImpl

public CloudStorageDataTransferApiImpl(JobInfo jobInfo,
                                       o.a.c.sidecar.client.shaded.client.SidecarClient sidecarClient,
                                       StorageClient storageClient,
                                       @Nullable
                                       java.lang.String clusterId)

Method Detail

jobInfo

public JobInfo jobInfo()

sidecarClient

public o.a.c.sidecar.client.shaded.client.SidecarClient sidecarClient()

uploadBundle

public BundleStorageObject uploadBundle(StorageCredentials writeCredentials,
                                        Bundle bundle)
                                 throws org.apache.cassandra.spark.exception.S3ApiCallException

Specified by:

uploadBundle in interface CloudStorageDataTransferApi

Throws:

org.apache.cassandra.spark.exception.S3ApiCallException

createRestoreJob

public void createRestoreJob(o.a.c.sidecar.client.shaded.common.request.data.CreateRestoreJobRequestPayload createRestoreJobRequestPayload)
                      throws org.apache.cassandra.spark.exception.SidecarApiCallException

Specified by:

createRestoreJob in interface CloudStorageDataTransferApi

Throws:

org.apache.cassandra.spark.exception.SidecarApiCallException

restoreJobSummary

public o.a.c.sidecar.client.shaded.common.response.data.RestoreJobSummaryResponsePayload restoreJobSummary()
                                                                                                    throws org.apache.cassandra.spark.exception.SidecarApiCallException

Specified by:

restoreJobSummary in interface CloudStorageDataTransferApi

Throws:

org.apache.cassandra.spark.exception.SidecarApiCallException

createRestoreSliceFromExecutor

public void createRestoreSliceFromExecutor(o.a.c.sidecar.client.shaded.client.SidecarInstance sidecarInstance,
                                           o.a.c.sidecar.client.shaded.common.request.data.CreateSliceRequestPayload createSliceRequestPayload)
                                    throws org.apache.cassandra.spark.exception.SidecarApiCallException

Description copied from interface: CloudStorageDataTransferApi

Called from task level to create a restore slice. The request retries until the slice is created (201) or retry has exhausted.

Specified by:

createRestoreSliceFromExecutor in interface CloudStorageDataTransferApi

Parameters:

sidecarInstance - the sidecar instance where we will create the slice

createSliceRequestPayload - the payload to create the slice

Throws:

org.apache.cassandra.spark.exception.SidecarApiCallException - when an error occurs during the slice creation

createRestoreSliceFromDriver

public java.util.concurrent.CompletableFuture<java.lang.Void> createRestoreSliceFromDriver(o.a.c.sidecar.client.shaded.client.SidecarInstance sidecarInstance,
                                                                                           o.a.c.sidecar.client.shaded.common.request.data.CreateSliceRequestPayload createSliceRequestPayload)

Description copied from interface: CloudStorageDataTransferApi

Called from driver level to create a restore slice asynchronously. The request retries until the slice succeeds (200), failed (550) or retry has exhausted.

Specified by:

createRestoreSliceFromDriver in interface CloudStorageDataTransferApi

Parameters:

sidecarInstance - the sidecar instance where we will create the slice

createSliceRequestPayload - the payload to create the slice

Returns:

future of create restore slice request

updateRestoreJob

public void updateRestoreJob(o.a.c.sidecar.client.shaded.common.request.data.UpdateRestoreJobRequestPayload updateRestoreJobRequestPayload)
                      throws org.apache.cassandra.spark.exception.SidecarApiCallException

Specified by:

updateRestoreJob in interface CloudStorageDataTransferApi

Throws:

org.apache.cassandra.spark.exception.SidecarApiCallException

abortRestoreJob

public void abortRestoreJob()
                     throws org.apache.cassandra.spark.exception.SidecarApiCallException

Specified by:

abortRestoreJob in interface CloudStorageDataTransferApi

Throws:

org.apache.cassandra.spark.exception.SidecarApiCallException