nrel.hive.util.iterators

Module Contents

Classes

ObjectIterator	iterator that deals with a set of named tuples
DictReaderIterator	iterator used internally by DictReaderStepper
DictReaderStepper	takes a DictReader and steps through it, using one specific column's values as a way to split

Functions

sliding(→ Generator)

iterate a sliding window over some iterable with a fixed window size

Attributes

log

nrel.hive.util.iterators.log

class nrel.hive.util.iterators.ObjectIterator(items: Tuple[Any, Ellipsis], step_attr_name: str, stop_condition: Callable)

iterator that deals with a set of named tuples

update_stop_condition(stop_condition: Callable)

__iter__()

__next__()

class nrel.hive.util.iterators.DictReaderIterator(reader: Iterator[Dict[str, str]], step_column_name: str, stop_condition: Callable, parser: Callable)

iterator used internally by DictReaderStepper

update_stop_condition(stop_condition: Callable)

__iter__()

__next__()

class nrel.hive.util.iterators.DictReaderStepper(dict_reader: Iterator[Dict[str, str]], file_reference: TextIO | None, step_column_name: str, initial_stop_condition: Callable = lambda x: ..., parser: Callable = lambda x: ...)

takes a DictReader and steps through it, using one specific column’s values as a way to split iteration over windows (of time, or other).

read_until_value consumes the next set of rows that fall within the next upper-value for the next window.

destruction: should be explicitly closed via DictReaderStepper.close()

classmethod build(file: str | pathlib.Path, step_column_name: str, initial_stop_condition: Callable = lambda x: ..., parser: Callable = lambda x: ...) → Tuple[Exception | None, DictReaderStepper | None]

alternative constructor that takes a file path and returns a DictReaderStepper, or, a failure

Parameters:

• file – the file path

• step_column_name – the column we are comparing new bounds against

• initial_stop_condition – the initial bounds - set low (zero) for ascending, high (inf) for descending note: descending not yet implemented

• parser – an optional parameter for parsing the input_config value

Returns:

a new reader or an exception

classmethod from_iterator(data: Iterator[Dict[str, str]], step_column_name: str, initial_stop_condition: Callable = lambda x: ..., parser: Callable = lambda x: ...) → DictReaderStepper

allows for substituting a simple Dict Iterator in place of loading from a file, allowing for programmatic data loading (for debugging, or, for dealing with default file contents)

Parameters:

• data – a provider of row-wise data similar to a CSV

• step_column_name – the key we are expecting in each Dict that we are comparing new bounds against

• initial_stop_condition – the initial bounds - set low (zero) for ascending, high (inf) for descending note: descending not yet implemented

• parser – an optional parameter for parsing the input_config value

Returns:

a new reader or an exception

read_until_stop_condition(stop_condition: Callable) → Iterator[Dict[str, str]]

reads rows from the DictReader as long as step_column_name is less than or equal to “value”

Parameters:

stop_condition – the condition to validate. we will read all new rows as long as each row’s value evaluates to True

Returns:

the updated DictReaderStepper and a tuple of rows, which may be empty if no new rows are consumable.

close()

nrel.hive.util.iterators.sliding(iterable: Iterable, size: int) → Generator

iterate a sliding window over some iterable with a fixed window size taken from [[https://codereview.stackexchange.com/a/239386]] :param iterable: the iterable to traverse :param size: the window size for sliding :return: an iterable of sliding windows over the original iterator