nip.parameters.agents.AgentParameters

class nip.parameters.agents.AgentParameters(agent_lr_factor: ~nip.parameters.agents.LrFactors | dict | None = None, body_lr_factor: ~nip.parameters.agents.LrFactors | dict | None = None, update_schedule: ~nip.parameters.update_schedule.AgentUpdateSchedule = ConstantUpdateSchedule(), use_manual_architecture: bool = False, normalize_message_history: bool = False, load_checkpoint_and_parameters: bool = False, checkpoint_entity: str = <factory>, checkpoint_project: str = <factory>, checkpoint_run_id: str | None = None, checkpoint_version: str = 'latest', use_orthogonal_initialisation: bool = True, orthogonal_initialisation_gain: float = 1.0)

Base class for sub-parameters objects which define agents.

Parameters:

• agent_lr_factor ([LrFactors | dict], optional) – The learning rate factor for the whole agent (split across the actor and the critic) compared with the base learning rate. This allows updating the agents at different rates.

• body_lr_factor ([LrFactors | dict], optional) – The learning rate factor for the body part of the model (split across the actor and the critic) compared with with whole agent. This allows updating the body at a different rate to the rest of the model.

• update_schedule (AgentUpdateSchedule) – The schedule for updating the agent weights when doing multi-agent training. This specifies on which iterations the agent should be updated by the optimizer.

• use_manual_architecture (bool) – Whether to use a manually defined architecture for the agent, which implements a hand-specified (non-learned) algorithm designed to maximise reward. This algorithm can be different depending on the environment. This is useful to test if the other agents can learn to work with a fixed optimum agent. It usually makes sense to set agent_lr_factor to {“actor”: 0, “critic”: 0} in this case.

• normalize_message_history (bool) – Whether to normalise the message history before passing it through the GNN encoder. Message histories are normalised to have zero mean and unit variance assuming that all episode lengths are equally frequent. (While this is probably not a realistic assumption, it’s the most reasonable one we can make without knowing the true distribution of episode lengths. It’s unlikely to make a big difference to the normalisation, and it’s probably better than not normalising at all.)

• load_checkpoint_and_parameters (bool) – Whether to load the agent model checkpoint and parameters from W&B. In this case, all agent parameters are replaced by the parameters from the checkpoint. Otherwise, the model is randomly initialised. If True, the checkpoint_run_id parameter must be set.

• checkpoint_entity (str, optional) – The entity of the W&B run to load the checkpoint from. If not provided, the default is used.

• checkpoint_project (str, optional) – The project of the W&B run to load the checkpoint from. If not provided, the default is used.

• checkpoint_run_id (str, optional) – The ID of the W&B run to load the checkpoint from. Must be provided if load_checkpoint is True.

• checkpoint_version (str) – The version of the checkpoint to load. If not provided, the latest version is used.

• use_orthogonal_initialisation (bool) – Whether to use orthogonal initialisation for the weights of the various networks.

• orthogonal_initialisation_gain (float) – The gain when using orthogonal initialisation.

Methods Summary

__eq__(other)	Return self==value.
__init__([agent_lr_factor, body_lr_factor, ...])
__post_init__()
__repr__()	Return repr(self).
_get_param_class_from_dict(param_dict)	Try to get the parameter class from a dictionary of serialised parameters.
construct_test_params()	Construct a set of basic parameters for testing.
from_dict(params_dict[, ignore_extra_keys])	Create a parameters object from a dictionary.
get(address)	Get a value from the parameters object using a dot-separated address.
load_from_wandb_config(wandb_config)	Load the parameters from a W&B config dictionary.
to_dict()	Convert the parameters object to a dictionary.

Attributes

LOAD_PRESERVED_PARAMETERS
agent_lr_factor
body_lr_factor
checkpoint_run_id
checkpoint_version
is_random
load_checkpoint_and_parameters
normalize_message_history
orthogonal_initialisation_gain
update_schedule
use_manual_architecture
use_orthogonal_initialisation
checkpoint_entity
checkpoint_project

Methods

__eq__(other)

Return self==value.

__init__(agent_lr_factor: ~nip.parameters.agents.LrFactors | dict | None = None, body_lr_factor: ~nip.parameters.agents.LrFactors | dict | None = None, update_schedule: ~nip.parameters.update_schedule.AgentUpdateSchedule = ConstantUpdateSchedule(), use_manual_architecture: bool = False, normalize_message_history: bool = False, load_checkpoint_and_parameters: bool = False, checkpoint_entity: str = <factory>, checkpoint_project: str = <factory>, checkpoint_run_id: str | None = None, checkpoint_version: str = 'latest', use_orthogonal_initialisation: bool = True, orthogonal_initialisation_gain: float = 1.0) → None

__post_init__()

__repr__()

Return repr(self).

classmethod _get_param_class_from_dict(param_dict: dict) → type[ParameterValue] | None

Try to get the parameter class from a dictionary of serialised parameters.

Parameters:

param_dict (dict) – A dictionary of parameters, which may have come from a to_dict method. This dictionary may contain a _type key, which is used to determine the class of the parameter.

Returns:

param_class (type[ParameterValue] | None) – The class of the parameter, if it can be determined.

Raises:

ValueError – If the class specified in the dictionary is not a valid parameter class.

classmethod construct_test_params() → BaseHyperParameters

Construct a set of basic parameters for testing.

classmethod from_dict(params_dict: dict, ignore_extra_keys: bool = False) → AgentsParameters

Create a parameters object from a dictionary.

Parameters:

• params_dict (dict) – A dictionary of the parameters.

• ignore_extra_keys (bool, default=False) – If True, ignore keys in the dictionary that do not correspond to fields in the parameters object.

Returns:

hyper_params (AgentsParameters) – The parameters object.

get(address: str) → Any

Get a value from the parameters object using a dot-separated address.

Parameters:

address (str) – The path to the value in the parameters object, separated by dots.

Returns:

value (Any) – The value at the address.

Raises:

KeyError – If the address does not exist.

load_from_wandb_config(wandb_config: dict)

Load the parameters from a W&B config dictionary.

Parameters:

wandb_config (dict) – The W&B config dictionary for this agent (e.g. wandb_run.config["agents"][agent_name]).

to_dict() → dict

Convert the parameters object to a dictionary.

Adds the is_random parameter to the dictionary. This is not a field of the parameters object, but we want to include it in the dictionary for logging.

Returns:

params_dict (dict) – A dictionary of the parameters.