Skip to content

Latest commit

 

History

History
57 lines (41 loc) · 1.96 KB

spark-sql-streaming-StateStoreCoordinatorRef.adoc

File metadata and controls

57 lines (41 loc) · 1.96 KB

StateStoreCoordinatorRef Interface for Communication with StateStoreCoordinator

StateStoreCoordinatorRef allows for communication with StateStoreCoordinator (through rpcEndpointRef reference).

Table 1. StateStoreCoordinatorRef’s Methods and Underlying RPC Messages (in alphabetical order)
Method RPC Message Description

deactivateInstances

DeactivateInstances

Synchronous event to announce that StreamingQueryManager has been informed that a query terminated (which is when StreamExecution has finished (running streaming batches)).

Note
Refer to DeactivateInstances (of StateStoreCoordinator) to know how the event is handled.

getLocation

GetLocation

reportActiveInstance

ReportActiveInstance

stop

StopCoordinator

verifyIfInstanceActive

VerifyIfInstanceActive

Creating StateStoreCoordinatorRef with StateStoreCoordinator RPC Endpoint — forDriver Method

forDriver(env: SparkEnv): StateStoreCoordinatorRef

forDriver…​FIXME

Note
forDriver is used exclusively when StreamingQueryManager is created.

forExecutor Method

forExecutor(env: SparkEnv): StateStoreCoordinatorRef

forExecutor…​FIXME

Note
forExecutor is used exclusively when StateStore creates a StateStoreCoordinatorRef (for executors).