Experiment Hyper-Parameters (`nip.parameters`)#

Overview#

The hyper-parameters object is the main API for the library. Hyper-parameters determine everything about how an experiment is run: the task, trainer, dataset and all parameters. An experiment should be completely reproducible from its hyper-parameters, up to hardware quirks and model API non-reproducibility.

A HyperParameters object is a nested dataclass. All configuration options (parameters) are listed in its documentation. Some parameters are themselves collections of sub-parameters. A HyperParameters object thus has a hierarchical (or nested) structure. All hyper-parameter and sub-parameter classes have BaseHyperParameters as base.

When creating sub-parameters, you can either pass then as an object of the appropriate SubParameters class, or as a dictionary. The advantage of the former is that you can use symbol inspection (e.g. in VS Code) to have easy access to the parameter names and descriptions. If you pass a dictionary, it will be converted to the appropriate SubParameters class.

Examples#

Create a parameters object, using default values for PPO parameters, and others

from nip.parameters import HyperParameters, AgentsParams, GraphIsomorphismAgentParameters

hyper_params = HyperParameters(
   scenario="graph_isomorphism",
   trainer="ppo",
   dataset="eru10000",
   agents=AgentsParams(
         prover=GraphIsomorphismAgentParameters(d_gnn=128),
         verifier=GraphIsomorphismAgentParameters(num_gnn_layers=2),
   ),
)

Convert the parameters object to a dictionary

>>> hyper_params.to_dict()
{'scenario': 'graph_isomorphism', 'trainer': 'ppo', 'dataset': 'eru10000', ...}

Create a parameters object using a dictionary for the ppo parameters

from nip.parameters import HyperParameters

hyper_params = HyperParameters(
   scenario="graph_isomorphism",
   trainer="ppo",
   dataset="eru10000",
   ppo={
         "num_epochs": 100,
         "batch_size": 256,
   },
)

Specifying Agent Parameters#

The number and names of the agents in the experiment vary depending on the protocol. Therefore, the sub-object which specifies the parameters for each agent is a special kind. The AgentsParameters class is a subclass of dict. The keys are the agent names, and the values are the parameters for that agent.

The special key "_default" can be used to specify default parameters for all agents. This is mostly useful in tests, where we want to use basic parameters for all agents, but don’t want to specify the names of all agents.

Converting to and from Nested Dicts#

A HyperParameters object can be converted to a nested dictionary. Example uses of this are attaching the hyper-parameters to a Weights & Biases run, and serialising the parameters to store them in a JSON file. To convert HyperParameters object to a dict, use the to_dict method (which is also available for sub-parameters). This performs the following special operations:

Some parameter values are not serialisable to JSON (e.g. AgentUpdateSchedule). These are converted to special dictionaries that can be converted back to the original object later.
Agents get the special key is_random added to their dictionary. This is a convenient way to see just from the dict if the agent selects their actions uniformly at random.

To convert a nested dictionary to a HyperParameters object, use the from_dict class method. This will reconstruct the original object, including all sub-parameters.

Creating New Parameters#

New parameters can be added by adding elements to the HyperParameters class or any of the sub-parameters classes. To a new sub-parameters class, just subclass SubParameters and decorate it with register_parameter_class to register it, and dataclass to make it a dataclass, as shown in the following example:

from nip.parameters import SubParameters, register_parameter_class
from dataclasses import dataclass

@register_parameter_class
@dataclass
class MyNewParameters(SubParameters):
    my_param: int = 10

Main Hyper-Parameters Class#

class nip.parameters.HyperParameters(scenario: Literal['graph_isomorphism', 'image_classification', 'code_validation'], trainer: Literal['vanilla_ppo', 'solo_agent', 'spg', 'reinforce', 'pure_text_ei', 'pure_text_malt'], dataset: str, interaction_protocol: Literal['nip', 'adp', 'debate', 'merlin_arthur', 'mnip', 'solo_verifier', 'multi_channel_test'] = 'nip', seed: int = 6198, functionalize_modules: bool = False, pretrain_agents: bool = False, test_size: float = 0.2, d_representation: int = 16, message_size: int = 1, include_linear_message_space: bool = False, d_linear_message_space: int = 2, agents: AgentsParameters | dict[str, AgentParameters] | None = None, rl: RlTrainerParameters | dict | None = None, ppo: CommonPpoParameters | dict | None = None, vanilla_ppo: VanillaPpoParameters | dict | None = None, spg: SpgParameters | dict | None = None, reinforce: ReinforceParameters | dict | None = None, solo_agent: SoloAgentParameters | dict | None = None, text_rl: TextRlParameters | dict | None = None, pure_text_ei: PureTextEiParameters | dict | None = None, pure_text_malt: PureTextMaltParameters | dict | None = None, image_classification: ImageClassificationParameters | dict | None = None, code_validation: CodeValidationParameters | dict | None = None, dataset_options: DatasetParameters | dict | None = None, protocol_common: CommonProtocolParameters | dict | None = None, nip_protocol: NipProtocolParameters | dict | None = None, debate_protocol: DebateProtocolParameters | dict | None = None, mnip_protocol: MnipProtocolParameters | dict | None = None, zk_protocol: ZkProtocolParameters | dict | None = None, message_regression: MessageRegressionParameters | dict | None = None, base_run: BaseRunParameters | dict | None = None)[source]#

The hyper-parameters of the experiment.

An experiment should be completely reproducible from its hyper-parameters (up to hardware quirks).

Parameters:

scenario (ScenarioType) – The name of the scenario to run, which specifies the domain, task and agents.
trainer (TrainerType) – The RL trainer to use.
dataset (str) – The dataset to use.
interaction_protocol (InteractionProtocolType) – The interaction protocol between the agents.
seed (int) – The random seed.
functionalize_modules (bool) – Whether to functionalize the modules in the agents. This allows some additional features which we don’t currently use, and comes with a big speed cost. Disabling it also prevents batch norm from tracking running statistics in eval mode, which might have a small effect on performance (unknown). Furthermore, disabling this prevents freezing parameters using requires_grad when doing a non-constant agent update schedule. Otherwise we get “RuntimeError: LSE is not correctly aligned”.
pretrain_agents (bool) – Whether to pretrain the agents in isolation before running the main training. This pretrains the bodies of the agents using the parameters in solo_agent.
test_size (float) – The proportion of the dataset to use for testing.
d_representation (int) – The dimension of each agent’s body representation output.
message_size (int) – The size of the message sent by agents. This is a dimension of the message space and effectively allows sending multiple messages simultaneously.
include_linear_message_space (bool) – Whether to include a 1-dimensional message space in addition to the message space specified by the scenario. This allows sending a single number as a message, in addition to the normal message. This can be useful for debugging issues with the message space, especially when combined with shared reward, since it should be easier to learn to send a single number when both agents want to cooperate.
d_linear_message_space (int) – The dimension of the linear message space (i.e. the number of possible messages which can sent). This is only used if include_linear_message_space is True.
agents (AgentsParameters | dict[str, AgentParameters], optional) – Parameters for the agents. The keys are the names of the agents, and the values are the parameters for each agent. If not provided, the default parameters are used for each agent for a given scenario.
rl (RlTrainerParams, optional) – Common parameters for all RL trainers.
ppo (PpoParameters, optional) – Common parameters for PPO trainers.
vanilla_ppo (VanillaPpoParameters, optional) – Parameters for the vanilla PPO trainer.
spg (SpgParameters, optional) – Parameters for SPG and its variants.
reinforce (ReinforceParameters, optional) – Parameters for the REINFORCE trainer.
solo_agent (SoloAgentParameters, optional) – Parameters for running agents in isolation. Used when the trainer is “solo_agent” or when pretrain_agents is True.
pure_text_ei (PureTextEiParameters, optional) – Parameters for the expert iteration (EI) trainer which works with agents that call a text-based API.
pure_text_malt (PureTextMaltParameters, optional) – Parameters for the multi-agent LLM training (MALT) trainer which works with agents that call a text-based API.
image_classification (ImageClassificationParameters, optional) – Parameters for the image classification task.
code_validation (CodeValidationParameters, optional) – Parameters for the code validation task.
dataset_options (DatasetParameters, optional) – Parameters for the dataset.
protocol_common (CommonProtocolParameters, optional) – Parameters common to all protocols.
nip_protocol (NipProtocolParameters, optional) – Parameters for the NIP protocol.
debate_protocol (DebateProtocolParameters, optional) – Parameters for the debate protocol.
mnip_protocol (MnipProtocolParameters, optional) – Parameters for the MNIP protocol.
zk_protocol (ZkProtocolParameters, optional) – Parameters for zero-knowledge protocols.
message_regression (MessageRegressionParameters, optional) – Parameters for doing regression analysis on the messages.
base_run (BaseRunParameters, optional) – Parameters for basing the current experiment on a previous W&B run.

Methods

`__init__`(scenario, trainer, dataset[, ...])
`construct_test_params`()	Construct a set of basic parameters for testing.
`from_dict`(params_dict[, ignore_extra_keys])	Create a parameters object from a dictionary.
`get`(address)	Get a value from the parameters object using a dot-separated address.
`to_dict`([include_package_meta])	Convert the parameters object to a dictionary.

Modules for Sub-Parameters#

Sub-parameter classes are grouped into sub-modules of nip.parameters. Each module contains classes for a specific part of the hyper-parameters.

`protocol`	Parameters for the interaction protocols.
`trainers`	Parameters for the various ML trainers.
`agents`	Parameters which specify agent configurations.
`dataset`	Parameters for the dataset.
`scenario`	Parameters specific to each scenario.
`base_run`	Parameters for basing the current experiment on a previous W&B run.
`message_regression`	Parameters for doing regression analysis on the messages.
`update_schedule`	Parameters specifying the schedule for updating agents.

Bases Classes and Enum Types#

`parameters_base`	Base classes for parameters objects.
`types`	Enum types for experiment parameters.

Handling Parameter Versions#

version

A function for transforming old hyper-parameter dicts to new ones.

Registering Parameter Classes#

nip.parameters.register_parameter_class(cls: type[P]) → type[P][source]#

Parameters:: cls (type[P]) – The class to register.
Returns:: cls (type[P]) – The same class that was passed in.

Experiment Hyper-Parameters (nip.parameters)

Contents

Experiment Hyper-Parameters (`nip.parameters`)#

Overview#

Examples#

Specifying Agent Parameters#

Converting to and from Nested Dicts#

Creating New Parameters#

Main Hyper-Parameters Class#

Modules for Sub-Parameters#

Bases Classes and Enum Types#

Handling Parameter Versions#

Registering Parameter Classes#

Experiment Hyper-Parameters (nip.parameters)

Contents

Experiment Hyper-Parameters (nip.parameters)#

Overview#

Examples#

Specifying Agent Parameters#

Converting to and from Nested Dicts#

Creating New Parameters#

Main Hyper-Parameters Class#

Modules for Sub-Parameters#

Bases Classes and Enum Types#

Handling Parameter Versions#

Registering Parameter Classes#

Experiment Hyper-Parameters (`nip.parameters`)#