Experiment Hyper-Parameters (nip.parameters
)#
Overview#
The hyper-parameters object is the main API for the library. Hyper-parameters determine everything about how an experiment is run: the task, trainer, dataset and all parameters. An experiment should be completely reproducible from its hyper-parameters, up to hardware quirks and model API non-reproducibility.
A HyperParameters
object is a nested dataclass
.
All configuration options (parameters) are listed in its documentation. Some parameters
are themselves collections of sub-parameters. A HyperParameters
object thus has
a hierarchical (or nested) structure. All hyper-parameter and sub-parameter classes have
BaseHyperParameters
as
base.
When creating sub-parameters, you can either pass then as an object of the appropriate
SubParameters
class, or as a
dictionary. The advantage of the former is that you can use symbol inspection (e.g. in
VS Code) to have easy access to the parameter names and descriptions. If you pass a
dictionary, it will be converted to the appropriate SubParameters
class.
Examples#
Create a parameters object, using default values for PPO parameters, and others
from nip.parameters import HyperParameters, AgentsParams, GraphIsomorphismAgentParameters hyper_params = HyperParameters( scenario="graph_isomorphism", trainer="ppo", dataset="eru10000", agents=AgentsParams( prover=GraphIsomorphismAgentParameters(d_gnn=128), verifier=GraphIsomorphismAgentParameters(num_gnn_layers=2), ), )
Convert the parameters object to a dictionary
>>> hyper_params.to_dict() {'scenario': 'graph_isomorphism', 'trainer': 'ppo', 'dataset': 'eru10000', ...}
Create a parameters object using a dictionary for the ppo parameters
from nip.parameters import HyperParameters hyper_params = HyperParameters( scenario="graph_isomorphism", trainer="ppo", dataset="eru10000", ppo={ "num_epochs": 100, "batch_size": 256, }, )
Specifying Agent Parameters#
The number and names of the agents in the experiment vary depending on the protocol.
Therefore, the sub-object which specifies the parameters for each agent is a special
kind. The AgentsParameters
class is a
subclass of dict
. The keys are the agent names, and the values are the
parameters for that agent.
The special key "_default"
can be used to specify default parameters for all agents.
This is mostly useful in tests, where we want to use basic parameters for all agents,
but don’t want to specify the names of all agents.
Converting to and from Nested Dicts#
A HyperParameters
object can be converted to a nested dictionary. Example uses
of this are attaching the hyper-parameters to a Weights & Biases run, and
serialising the parameters to store them in a JSON file. To convert
HyperParameters
object to a dict, use the to_dict
method (which is also available for
sub-parameters). This performs the following special operations:
Some parameter values are not serialisable to JSON (e.g.
AgentUpdateSchedule
). These are converted to special dictionaries that can be converted back to the original object later.Agents get the special key
is_random
added to their dictionary. This is a convenient way to see just from the dict if the agent selects their actions uniformly at random.
To convert a nested dictionary to a HyperParameters
object, use the
from_dict
class method. This will
reconstruct the original object, including all sub-parameters.
Creating New Parameters#
New parameters can be added by adding elements to the HyperParameters
class or
any of the sub-parameters classes. To a new sub-parameters class, just subclass
SubParameters
and decorate it
with register_parameter_class
to register it, and
dataclass
to make it a dataclass, as shown in the
following example:
from nip.parameters import SubParameters, register_parameter_class
from dataclasses import dataclass
@register_parameter_class
@dataclass
class MyNewParameters(SubParameters):
my_param: int = 10
Main Hyper-Parameters Class#
- class nip.parameters.HyperParameters(scenario: Literal['graph_isomorphism', 'image_classification', 'code_validation'], trainer: Literal['vanilla_ppo', 'solo_agent', 'spg', 'reinforce', 'pure_text_ei', 'pure_text_malt'], dataset: str, interaction_protocol: Literal['nip', 'adp', 'debate', 'merlin_arthur', 'mnip', 'solo_verifier', 'multi_channel_test'] = 'nip', seed: int = 6198, functionalize_modules: bool = False, pretrain_agents: bool = False, test_size: float = 0.2, d_representation: int = 16, message_size: int = 1, include_linear_message_space: bool = False, d_linear_message_space: int = 2, agents: AgentsParameters | dict[str, AgentParameters] | None = None, rl: RlTrainerParameters | dict | None = None, ppo: CommonPpoParameters | dict | None = None, vanilla_ppo: VanillaPpoParameters | dict | None = None, spg: SpgParameters | dict | None = None, reinforce: ReinforceParameters | dict | None = None, solo_agent: SoloAgentParameters | dict | None = None, text_rl: TextRlParameters | dict | None = None, pure_text_ei: PureTextEiParameters | dict | None = None, pure_text_malt: PureTextMaltParameters | dict | None = None, image_classification: ImageClassificationParameters | dict | None = None, code_validation: CodeValidationParameters | dict | None = None, dataset_options: DatasetParameters | dict | None = None, protocol_common: CommonProtocolParameters | dict | None = None, nip_protocol: NipProtocolParameters | dict | None = None, debate_protocol: DebateProtocolParameters | dict | None = None, mnip_protocol: MnipProtocolParameters | dict | None = None, zk_protocol: ZkProtocolParameters | dict | None = None, message_regression: MessageRegressionParameters | dict | None = None, base_run: BaseRunParameters | dict | None = None)[source]#
The hyper-parameters of the experiment.
An experiment should be completely reproducible from its hyper-parameters (up to hardware quirks).
- Parameters:
scenario (ScenarioType) – The name of the scenario to run, which specifies the domain, task and agents.
trainer (TrainerType) – The RL trainer to use.
dataset (str) – The dataset to use.
interaction_protocol (InteractionProtocolType) – The interaction protocol between the agents.
seed (int) – The random seed.
functionalize_modules (bool) – Whether to functionalize the modules in the agents. This allows some additional features which we don’t currently use, and comes with a big speed cost. Disabling it also prevents batch norm from tracking running statistics in eval mode, which might have a small effect on performance (unknown). Furthermore, disabling this prevents freezing parameters using
requires_grad
when doing a non-constant agent update schedule. Otherwise we get “RuntimeError: LSE is not correctly aligned”.pretrain_agents (bool) – Whether to pretrain the agents in isolation before running the main training. This pretrains the bodies of the agents using the parameters in
solo_agent
.test_size (float) – The proportion of the dataset to use for testing.
d_representation (int) – The dimension of each agent’s body representation output.
message_size (int) – The size of the message sent by agents. This is a dimension of the message space and effectively allows sending multiple messages simultaneously.
include_linear_message_space (bool) – Whether to include a 1-dimensional message space in addition to the message space specified by the scenario. This allows sending a single number as a message, in addition to the normal message. This can be useful for debugging issues with the message space, especially when combined with shared reward, since it should be easier to learn to send a single number when both agents want to cooperate.
d_linear_message_space (int) – The dimension of the linear message space (i.e. the number of possible messages which can sent). This is only used if
include_linear_message_space
isTrue
.agents (AgentsParameters | dict[str, AgentParameters], optional) – Parameters for the agents. The keys are the names of the agents, and the values are the parameters for each agent. If not provided, the default parameters are used for each agent for a given scenario.
rl (RlTrainerParams, optional) – Common parameters for all RL trainers.
ppo (PpoParameters, optional) – Common parameters for PPO trainers.
vanilla_ppo (VanillaPpoParameters, optional) – Parameters for the vanilla PPO trainer.
spg (SpgParameters, optional) – Parameters for SPG and its variants.
reinforce (ReinforceParameters, optional) – Parameters for the REINFORCE trainer.
solo_agent (SoloAgentParameters, optional) – Parameters for running agents in isolation. Used when the trainer is “solo_agent” or when
pretrain_agents
isTrue
.pure_text_ei (PureTextEiParameters, optional) – Parameters for the expert iteration (EI) trainer which works with agents that call a text-based API.
pure_text_malt (PureTextMaltParameters, optional) – Parameters for the multi-agent LLM training (MALT) trainer which works with agents that call a text-based API.
image_classification (ImageClassificationParameters, optional) – Parameters for the image classification task.
code_validation (CodeValidationParameters, optional) – Parameters for the code validation task.
dataset_options (DatasetParameters, optional) – Parameters for the dataset.
protocol_common (CommonProtocolParameters, optional) – Parameters common to all protocols.
nip_protocol (NipProtocolParameters, optional) – Parameters for the NIP protocol.
debate_protocol (DebateProtocolParameters, optional) – Parameters for the debate protocol.
mnip_protocol (MnipProtocolParameters, optional) – Parameters for the MNIP protocol.
zk_protocol (ZkProtocolParameters, optional) – Parameters for zero-knowledge protocols.
message_regression (MessageRegressionParameters, optional) – Parameters for doing regression analysis on the messages.
base_run (BaseRunParameters, optional) – Parameters for basing the current experiment on a previous W&B run.
Methods
__init__
(scenario, trainer, dataset[, ...])construct_test_params
()Construct a set of basic parameters for testing.
from_dict
(params_dict[, ignore_extra_keys])Create a parameters object from a dictionary.
get
(address)Get a value from the parameters object using a dot-separated address.
to_dict
([include_package_meta])Convert the parameters object to a dictionary.
Modules for Sub-Parameters#
Sub-parameter classes are grouped into sub-modules of nip.parameters
. Each module
contains classes for a specific part of the hyper-parameters.
Parameters for the interaction protocols. |
|
Parameters for the various ML trainers. |
|
Parameters which specify agent configurations. |
|
Parameters for the dataset. |
|
Parameters specific to each scenario. |
|
Parameters for basing the current experiment on a previous W&B run. |
|
Parameters for doing regression analysis on the messages. |
|
Parameters specifying the schedule for updating agents. |
Bases Classes and Enum Types#
Base classes for parameters objects. |
|
Enum types for experiment parameters. |
Handling Parameter Versions#
A function for transforming old hyper-parameter dicts to new ones. |