nip.utils.nested_array_dict.NestedArrayDict#

class nip.utils.nested_array_dict.NestedArrayDict(data: Any | None = None, batch_size: int | Sequence[int] | None = None)[source]#

A nested dictionary of numpy arrays.

A NestedDict behaves similarly to a TensorDict: it allows for nested dictionaries and has a common batch size

Methods Summary

__contains__(key)

__getitem__(index)

__init__([data, batch_size])

__len__()

__repr__()

Return repr(self).

__setitem__(index, value)

_create_arrays_from_data(data, batch_size)

Create a list of named arrays from the data.

_is_shape_compatible(shape)

Check whether a shape is compatible with the batch size.

_make_key_repr(keys[, prefix])

Make a string representation for a set of keys.

clone([recurse])

Clone the NestedDict and create a new instance.

from_arrays_and_scalars(array_and_scalars, ...)

Create a NestedDict from a list of arrays and scalars, and associated keys.

items([include_prefixes, return_all_tuples])

Return an iterator over the items, optionally including prefixes.

key_is_leaf(key)

Check if a key is a leaf, i.e. it is not a prefix of any other key.

keys([include_prefixes, return_all_tuples])

Return an iterator over the all keys, optionally including prefixes.

leaf_keys()

Return the leaf keys as tuples.

update(other)

Update the NestedDict with the data from another dict.

zeros_like(other)

Create a NestedDict with zeros in the same structure as another NestedDict.

Attributes

batch_size

The batch size of the NestedDict.

Methods

__contains__(key: Any)[source]#
__getitem__(index: Any) NestedArrayDict | ndarray[Any, dtype[_ScalarType_co]][source]#
__init__(data: Any | None = None, batch_size: int | Sequence[int] | None = None)[source]#
__len__()[source]#
__repr__()[source]#

Return repr(self).

__setitem__(index: Any, value: Any)[source]#
_create_arrays_from_data(data: dict[str, list | _SupportsArray[dtype[Any]] | _NestedSequence[_SupportsArray[dtype[Any]]] | bool | int | float | complex | str | bytes | _NestedSequence[bool | int | float | complex | str | bytes] | Any] | Any, batch_size: tuple[int, ...]) tuple[list[tuple[str, ...]], list[ndarray[Any, dtype[_ScalarType_co]]], tuple[int, ...]][source]#

Create a list of named arrays from the data.

Returns a list of keys and a list of corresponding arrays.

The data are validated and flattened, so that each key is a tuple of strings.

Parameters:
  • data (dict) – The data to convert. Can be a nested dictionary of lists or arrays. All keys must be strings.

  • batch_size (tuple[int, ...]) – The batch size of the NestedDict, which must be the initial segment of the shape of the each array. If (), the arrays are scalars.

Returns:

  • keys (list[tuple[str, …]]) – The keys of the data. Each key is a tuple of strings that represents the path to the data in the nested dictionary.

  • arrays (list[NDArray]) – The data as numpy arrays. The order of the arrays corresponds to the order of the keys.

_is_shape_compatible(shape: tuple[int, ...]) bool[source]#

Check whether a shape is compatible with the batch size.

Parameters:

shape (tuple[int, ...]) – The shape to check.

Returns:

is_compatible (bool) – Whether the shape is compatible with the batch size.

_make_key_repr(keys: list[tuple[str, ...]], prefix: tuple[str, ...] = ()) str[source]#

Make a string representation for a set of keys.

Represents the keys as a nested NestedDict.

Parameters:
  • keys (list[tuple[str, ...]]) – The keys to represent.

  • prefix (tuple[str, ...], optional) – The prefix to add to the keys, to get the full path.

Returns:

key_repr (str) – The string representation of the keys.

clone(recurse: bool = True) NestedArrayDict[source]#

Clone the NestedDict and create a new instance.

Parameters:

recurse (bool, default=True) – Whether to clone the arrays and scalars in the NestedDict. Otherwise, just clone the structure.

Returns:

cloned_dict (NestedDict) – The cloned NestedDict.

classmethod from_arrays_and_scalars(array_and_scalars: list[_SupportsArray[dtype[Any]] | _NestedSequence[_SupportsArray[dtype[Any]]] | bool | int | float | complex | str | bytes | _NestedSequence[bool | int | float | complex | str | bytes] | Any], tuple_keys: list[tuple[str, ...]], batch_size: tuple[int, ...]) NestedArrayDict[source]#

Create a NestedDict from a list of arrays and scalars, and associated keys.

Parameters:
  • array_and_scalars (list[ArrayLike | Any]) – The list of arrays and scalars to store in the NestedDict.

  • tuple_keys (list[tuple[str, ...]]) – The keys of the data. Each key is a tuple of strings that represents the path to the data in the nested dictionary.

  • batch_size (tuple[int, ...]) – The batch size of the NestedDict, which must be the initial segment of the shape of the each array. If (), the arrays are scalars.

Returns:

string_dict (NestedDict) – The NestedDict created from the arrays.

items(include_prefixes: bool = True, return_all_tuples: bool = False) Iterator[tuple[tuple[str, ...], ndarray[Any, dtype[_ScalarType_co]]]][source]#

Return an iterator over the items, optionally including prefixes.

By default top-level keys are returned as strings, while nested keys are returned as tuples.

Parameters:
  • include_prefixes (bool, default=True) – Whether to include prefixes of the leaf keys.

  • return_all_tuples (bool, default=False) – If true, all keys will be returned as tuples. Otherwise, single element tuples will be replaced by their items

Yields:
  • key (tuple[str, …]) – The key of the item.

  • value (NDArray) – The value of the item.

key_is_leaf(key: tuple[str, ...]) bool[source]#

Check if a key is a leaf, i.e. it is not a prefix of any other key.

Leaf keys are keys that correspond to numpy arrays.

Parameters:

key (tuple[str, ...]) – The key to check.

Returns:

is_terminal (bool) – Whether the key is terminal.

keys(include_prefixes: bool = True, return_all_tuples: bool = False) Iterator[tuple[str, ...] | str][source]#

Return an iterator over the all keys, optionally including prefixes.

By default top-level keys are returned as strings, while nested keys are returned as tuples.

Parameters:
  • include_prefixes (bool, default=True) – Whether to include prefixes of the leaf keys.

  • return_all_tuples (bool, default=False) – If true, all keys will be returned as tuples. Otherwise, single element tuples will be replaced by their items

Yields:

key (tuple[str, …] | str) – The keys of the data. Each key is either a string or a a tuple of strings that represents the path to the data in the nested dictionary.

leaf_keys() list[tuple[str, ...]][source]#

Return the leaf keys as tuples.

Returns:

leaf_keys (list[tuple[str, …]]) – The leaf keys of the data. Each key is a tuple of strings that represents the path to the data in the nested dictionary to an array

update(other: NestedArrayDict | dict) NestedArrayDict[source]#

Update the NestedDict with the data from another dict.

Parameters:

other (NestedDict | dict) – The dict to update from. If a dict is provided, it is converted to a NestedDict with the same batch size as the current NestedDict.

Returns:

self (NestedDict) – The updated NestedDict.

classmethod zeros_like(other: NestedArrayDict) NestedArrayDict[source]#

Create a NestedDict with zeros in the same structure as another NestedDict.

Parameters:

other (NestedArrayDict) – The NestedArrayDict to copy the structure from.

Returns:

zero_dict (NestedArrayDict) – The NestedArrayDict with the same structure as other, but with zero arrays.