nip.utils.nested_array_dict.NestedArrayDict

class nip.utils.nested_array_dict.NestedArrayDict(data: Any | None = None, batch_size: int | Sequence[int] | None = None)

A nested dictionary of numpy arrays.

A NestedDict behaves similarly to a TensorDict: it allows for nested dictionaries and has a common batch size

Methods Summary

__contains__(key)
__getitem__(index)
__init__([data, batch_size])
__len__()
__repr__()	Return repr(self).
__setitem__(index, value)
_create_arrays_from_data(data, batch_size)	Create a list of named arrays from the data.
_is_shape_compatible(shape)	Check whether a shape is compatible with the batch size.
_make_key_repr(keys[, prefix])	Make a string representation for a set of keys.
clone([recurse])	Clone the NestedDict and create a new instance.
from_arrays_and_scalars(array_and_scalars, ...)	Create a NestedDict from a list of arrays and scalars, and associated keys.
items([include_prefixes, return_all_tuples])	Return an iterator over the items, optionally including prefixes.
key_is_leaf(key)	Check if a key is a leaf, i.e. it is not a prefix of any other key.
keys([include_prefixes, return_all_tuples])	Return an iterator over the all keys, optionally including prefixes.
leaf_keys()	Return the leaf keys as tuples.
update(other)	Update the NestedDict with the data from another dict.
zeros_like(other)	Create a NestedDict with zeros in the same structure as another NestedDict.

Attributes

batch_size

The batch size of the NestedDict.

Methods

__contains__(key: Any)

__getitem__(index: Any) → NestedArrayDict | ndarray[Any, dtype[_ScalarType_co]]

__init__(data: Any | None = None, batch_size: int | Sequence[int] | None = None)

__len__()

__repr__()

Return repr(self).

__setitem__(index: Any, value: Any)

_create_arrays_from_data(data: dict[str, list | _SupportsArray[dtype[Any]] | _NestedSequence[_SupportsArray[dtype[Any]]] | bool | int | float | complex | str | bytes | _NestedSequence[bool | int | float | complex | str | bytes] | Any] | Any, batch_size: tuple[int, ...]) → tuple[list[tuple[str, ...]], list[ndarray[Any, dtype[_ScalarType_co]]], tuple[int, ...]]

Create a list of named arrays from the data.

Returns a list of keys and a list of corresponding arrays.

The data are validated and flattened, so that each key is a tuple of strings.

Parameters:

• data (dict) – The data to convert. Can be a nested dictionary of lists or arrays. All keys must be strings.

• batch_size (tuple[int, ...]) – The batch size of the NestedDict, which must be the initial segment of the shape of the each array. If (), the arrays are scalars.

Returns:

• keys (list[tuple[str, …]]) – The keys of the data. Each key is a tuple of strings that represents the path to the data in the nested dictionary.

• arrays (list[NDArray]) – The data as numpy arrays. The order of the arrays corresponds to the order of the keys.

_is_shape_compatible(shape: tuple[int, ...]) → bool

Check whether a shape is compatible with the batch size.

Parameters:

shape (tuple[int, ...]) – The shape to check.

Returns:

is_compatible (bool) – Whether the shape is compatible with the batch size.

_make_key_repr(keys: list[tuple[str, ...]], prefix: tuple[str, ...] = ()) → str

Make a string representation for a set of keys.

Represents the keys as a nested NestedDict.

Parameters:

• keys (list[tuple[str, ...]]) – The keys to represent.

• prefix (tuple[str, ...], optional) – The prefix to add to the keys, to get the full path.

Returns:

key_repr (str) – The string representation of the keys.

clone(recurse: bool = True) → NestedArrayDict

Clone the NestedDict and create a new instance.

Parameters:

recurse (bool, default=True) – Whether to clone the arrays and scalars in the NestedDict. Otherwise, just clone the structure.

Returns:

cloned_dict (NestedDict) – The cloned NestedDict.

classmethod from_arrays_and_scalars(array_and_scalars: list[_SupportsArray[dtype[Any]] | _NestedSequence[_SupportsArray[dtype[Any]]] | bool | int | float | complex | str | bytes | _NestedSequence[bool | int | float | complex | str | bytes] | Any], tuple_keys: list[tuple[str, ...]], batch_size: tuple[int, ...]) → NestedArrayDict

Create a NestedDict from a list of arrays and scalars, and associated keys.

Parameters:

• array_and_scalars (list[ArrayLike | Any]) – The list of arrays and scalars to store in the NestedDict.

• tuple_keys (list[tuple[str, ...]]) – The keys of the data. Each key is a tuple of strings that represents the path to the data in the nested dictionary.

• batch_size (tuple[int, ...]) – The batch size of the NestedDict, which must be the initial segment of the shape of the each array. If (), the arrays are scalars.

Returns:

string_dict (NestedDict) – The NestedDict created from the arrays.

items(include_prefixes: bool = True, return_all_tuples: bool = False) → Iterator[tuple[tuple[str, ...], ndarray[Any, dtype[_ScalarType_co]]]]

Return an iterator over the items, optionally including prefixes.

By default top-level keys are returned as strings, while nested keys are returned as tuples.

Parameters:

• include_prefixes (bool, default=True) – Whether to include prefixes of the leaf keys.

• return_all_tuples (bool, default=False) – If true, all keys will be returned as tuples. Otherwise, single element tuples will be replaced by their items

Yields:

• key (tuple[str, …]) – The key of the item.

• value (NDArray) – The value of the item.

key_is_leaf(key: tuple[str, ...]) → bool

Check if a key is a leaf, i.e. it is not a prefix of any other key.

Leaf keys are keys that correspond to numpy arrays.

Parameters:

key (tuple[str, ...]) – The key to check.

Returns:

is_terminal (bool) – Whether the key is terminal.

keys(include_prefixes: bool = True, return_all_tuples: bool = False) → Iterator[tuple[str, ...] | str]

Return an iterator over the all keys, optionally including prefixes.

By default top-level keys are returned as strings, while nested keys are returned as tuples.

Parameters:

• include_prefixes (bool, default=True) – Whether to include prefixes of the leaf keys.

• return_all_tuples (bool, default=False) – If true, all keys will be returned as tuples. Otherwise, single element tuples will be replaced by their items

Yields:

key (tuple[str, …] | str) – The keys of the data. Each key is either a string or a a tuple of strings that represents the path to the data in the nested dictionary.

leaf_keys() → list[tuple[str, ...]]

Return the leaf keys as tuples.

Returns:

leaf_keys (list[tuple[str, …]]) – The leaf keys of the data. Each key is a tuple of strings that represents the path to the data in the nested dictionary to an array

update(other: NestedArrayDict | dict) → NestedArrayDict

Update the NestedDict with the data from another dict.

Parameters:

other (NestedDict | dict) – The dict to update from. If a dict is provided, it is converted to a NestedDict with the same batch size as the current NestedDict.

Returns:

self (NestedDict) – The updated NestedDict.

classmethod zeros_like(other: NestedArrayDict) → NestedArrayDict

Create a NestedDict with zeros in the same structure as another NestedDict.

Parameters:

other (NestedArrayDict) – The NestedArrayDict to copy the structure from.

Returns:

zero_dict (NestedArrayDict) – The NestedArrayDict with the same structure as other, but with zero arrays.