Skip to content

Latest commit

 

History

History
43 lines (33 loc) · 947 Bytes

spark-sql-streaming-ForeachWriter.adoc

File metadata and controls

43 lines (33 loc) · 947 Bytes

ForeachWriter

ForeachWriter is the contract for a foreach writer that is a streaming format that controls streaming writes.

Note
ForeachWriter is set using foreach operator.
val foreachWriter = new ForeachWriter[String] { ... }
streamingQuery.
  writeStream.
  foreach(foreachWriter).
  start

ForeachWriter Contract

package org.apache.spark.sql

abstract class ForeachWriter[T] {
  def open(partitionId: Long, version: Long): Boolean
  def process(value: T): Unit
  def close(errorOrNull: Throwable): Unit
}
Table 1. ForeachWriter Contract
Method Description

open

Used when…​

process

Used when…​

close

Used when…​