Base interface

Bases: ABC

Representation of readable file format.

Added in 0.9.0

check_if_supported(spark) abstractmethod

Check if Spark session does support this file format.

Added in 0.9.0

Raises:

• RuntimeError –

  If file format is not supported.

apply_to_reader(reader) abstractmethod

Apply provided format to pyspark.sql.DataFrameReader.

Added in 0.9.0

Returns:

• `pyspark.sql.DataFrameReader` –

  DataFrameReader with options applied.

Bases: ABC

Representation of writable file format.

Added in 0.9.0

check_if_supported(spark) abstractmethod

Check if Spark session does support this file format.

Added in 0.9.0

Raises:

• RuntimeError –

  If file format is not supported.

apply_to_writer(writer) abstractmethod

Apply provided format to pyspark.sql.DataFrameWriter.

Added in 0.9.0

Returns:

• `pyspark.sql.DataFrameWriter` –

  DataFrameWriter with options applied.