File Mover Options

Bases: GenericOptions

File moving options.

Added in 0.8.0

Examples:

from onetl.file import FileMover

options = FileMover.Options(
    if_exists="replace_entire_directory",
    workers=4,
)

if_exists = Field(default=(FileExistBehavior.ERROR), alias=(avoid_alias('mode'))) class-attribute instance-attribute

How to handle existing files in the local directory.

Possible values:

• error (default) - mark file as failed
• ignore - mark file as skipped
• replace_file - replace existing file with a new one
• replace_entire_directory - delete directory content before moving files

Added in 0.8.0

Changed in 0.9.0

Renamed mode → if_exists

workers = Field(default=1, ge=1) class-attribute instance-attribute

Number of workers to create for parallel file moving.

1 (default) means files will me moved sequentially. 2 or more means files will be moved in parallel workers.

Recommended value is min(32, os.cpu_count() + 4), e.g. 5.

Added in 0.8.1