nip.parameters.agents.AgentParameters#
- class nip.parameters.agents.AgentParameters(agent_lr_factor: ~nip.parameters.agents.LrFactors | dict | None = None, body_lr_factor: ~nip.parameters.agents.LrFactors | dict | None = None, update_schedule: ~nip.parameters.update_schedule.AgentUpdateSchedule = ConstantUpdateSchedule(), use_manual_architecture: bool = False, normalize_message_history: bool = False, load_checkpoint_and_parameters: bool = False, checkpoint_entity: str = <factory>, checkpoint_project: str = <factory>, checkpoint_run_id: str | None = None, checkpoint_version: str = 'latest', use_orthogonal_initialisation: bool = True, orthogonal_initialisation_gain: float = 1.0)[source]#
Base class for sub-parameters objects which define agents.
- Parameters:
agent_lr_factor ([LrFactors | dict], optional) – The learning rate factor for the whole agent (split across the actor and the critic) compared with the base learning rate. This allows updating the agents at different rates.
body_lr_factor ([LrFactors | dict], optional) – The learning rate factor for the body part of the model (split across the actor and the critic) compared with with whole agent. This allows updating the body at a different rate to the rest of the model.
update_schedule (AgentUpdateSchedule) – The schedule for updating the agent weights when doing multi-agent training. This specifies on which iterations the agent should be updated by the optimizer.
use_manual_architecture (bool) – Whether to use a manually defined architecture for the agent, which implements a hand-specified (non-learned) algorithm designed to maximise reward. This algorithm can be different depending on the environment. This is useful to test if the other agents can learn to work with a fixed optimum agent. It usually makes sense to set
agent_lr_factor
to {“actor”: 0, “critic”: 0} in this case.normalize_message_history (bool) – Whether to normalise the message history before passing it through the GNN encoder. Message histories are normalised to have zero mean and unit variance assuming that all episode lengths are equally frequent. (While this is probably not a realistic assumption, it’s the most reasonable one we can make without knowing the true distribution of episode lengths. It’s unlikely to make a big difference to the normalisation, and it’s probably better than not normalising at all.)
load_checkpoint_and_parameters (bool) – Whether to load the agent model checkpoint and parameters from W&B. In this case, all agent parameters are replaced by the parameters from the checkpoint. Otherwise, the model is randomly initialised. If
True
, thecheckpoint_run_id
parameter must be set.checkpoint_entity (str, optional) – The entity of the W&B run to load the checkpoint from. If not provided, the default is used.
checkpoint_project (str, optional) – The project of the W&B run to load the checkpoint from. If not provided, the default is used.
checkpoint_run_id (str, optional) – The ID of the W&B run to load the checkpoint from. Must be provided if
load_checkpoint
isTrue
.checkpoint_version (str) – The version of the checkpoint to load. If not provided, the latest version is used.
use_orthogonal_initialisation (bool) – Whether to use orthogonal initialisation for the weights of the various networks.
orthogonal_initialisation_gain (float) – The gain when using orthogonal initialisation.
Methods Summary
__eq__
(other)Return self==value.
__init__
([agent_lr_factor, body_lr_factor, ...])__repr__
()Return repr(self).
_get_param_class_from_dict
(param_dict)Try to get the parameter class from a dictionary of serialised parameters.
Construct a set of basic parameters for testing.
from_dict
(params_dict[, ignore_extra_keys])Create a parameters object from a dictionary.
get
(address)Get a value from the parameters object using a dot-separated address.
load_from_wandb_config
(wandb_config)Load the parameters from a W&B config dictionary.
to_dict
()Convert the parameters object to a dictionary.
Attributes
LOAD_PRESERVED_PARAMETERS
agent_lr_factor
body_lr_factor
checkpoint_run_id
checkpoint_version
is_random
load_checkpoint_and_parameters
normalize_message_history
orthogonal_initialisation_gain
update_schedule
use_manual_architecture
use_orthogonal_initialisation
checkpoint_entity
checkpoint_project
Methods
- __eq__(other)#
Return self==value.
- __init__(agent_lr_factor: ~nip.parameters.agents.LrFactors | dict | None = None, body_lr_factor: ~nip.parameters.agents.LrFactors | dict | None = None, update_schedule: ~nip.parameters.update_schedule.AgentUpdateSchedule = ConstantUpdateSchedule(), use_manual_architecture: bool = False, normalize_message_history: bool = False, load_checkpoint_and_parameters: bool = False, checkpoint_entity: str = <factory>, checkpoint_project: str = <factory>, checkpoint_run_id: str | None = None, checkpoint_version: str = 'latest', use_orthogonal_initialisation: bool = True, orthogonal_initialisation_gain: float = 1.0) None #
- __repr__()#
Return repr(self).
- classmethod _get_param_class_from_dict(param_dict: dict) type[ParameterValue] | None [source]#
Try to get the parameter class from a dictionary of serialised parameters.
- Parameters:
param_dict (dict) – A dictionary of parameters, which may have come from a
to_dict
method. This dictionary may contain a_type
key, which is used to determine the class of the parameter.- Returns:
param_class (type[ParameterValue] | None) – The class of the parameter, if it can be determined.
- Raises:
ValueError – If the class specified in the dictionary is not a valid parameter class.
- classmethod construct_test_params() BaseHyperParameters [source]#
Construct a set of basic parameters for testing.
- classmethod from_dict(params_dict: dict, ignore_extra_keys: bool = False) AgentsParameters [source]#
Create a parameters object from a dictionary.
- get(address: str) Any [source]#
Get a value from the parameters object using a dot-separated address.