Skip to content

Base interface

Bases: ABC

Base file limit interface.

Limits used by several onETL components, including File Downloader and File Mover, to determine if internal loop should be stopped.

Unlike file filters, limits have internal state which can be updated or reset.

Added in 0.8.0

is_reached abstractmethod property

Check if limit is reached.

Added in 0.8.0

Returns:

  • bool –

    True if limit is reached, False otherwise.

Examples:

>>> from onetl.impl import LocalPath
>>> limit.is_reached
False
>>> limit.stops_at(LocalPath("/path/to/file.csv"))
False
>>> limit.is_reached
False
>>> # after limit is reached
>>> limit.stops_at(LocalPath("/path/to/file.csv"))
True
>>> limit.is_reached
True

reset() abstractmethod

Resets the internal limit state.

Added in 0.8.0

Returns:

  • Self –

    Returns a filter of the same type, but with non-reached state.

    It could be the same filter or a new one, this is an implementation detail.

Examples:

>>> limit.is_reached
True
>>> new_limit = limit.reset()
>>> new_limit.is_reached
False

stops_at(path) abstractmethod

Update internal state and return current state.

Added in 0.8.0

Parameters:

  • path (PathProtocol) –

    Path to check

Returns:

  • bool –

    True if limit is reached, False otherwise.

Examples:

>>> from onetl.impl import LocalPath
>>> # limit is not reached yet
>>> limit.stops_at(LocalPath("/path/to/file.csv"))
False
>>> # after limit is reached
>>> limit.stops_at(LocalPath("/path/to/another.csv"))
True
>>> # at this point, .stops_at() and .is_reached will always return True,
>>> # even on inputs that returned False before.
>>> # it will be in the same state until .reset() is called
>>> limit.stops_at(LocalPath("/path/to/file.csv"))
True