nip.code_validation.agents.CodeValidationCombinedWholeAgent

class nip.code_validation.agents.CodeValidationCombinedWholeAgent(hyper_params: HyperParameters, settings: ExperimentSettings, protocol_handler: ProtocolHandler, wholes: dict[str, WholeAgent])

Module which combines all agents for code validation.

Methods Summary

__init__(hyper_params, settings, ...)
_expand_output_to_all_channels(agent_name, ...)	Expand an agent's output from its visible message channels to all.
_restrict_input_to_visible_channels(...)	Restrict an agent's input to its visible message channels.
forward(data, environment)	Run the forward pass through all agent parts and combine the outputs.

Attributes

additional_in_keys
additional_out_keys
excluded_in_keys
excluded_out_keys
in_keys	The keys required by the module.
out_keys	The keys produced by the module.

Methods

__init__(hyper_params: HyperParameters, settings: ExperimentSettings, protocol_handler: ProtocolHandler, wholes: dict[str, WholeAgent])

_expand_output_to_all_channels(agent_name: str, output: ndarray[Any, dtype[_ScalarType_co]], shape_spec: str, fill_value: Any = None) → ndarray[Any, dtype[_ScalarType_co]]

Expand an agent’s output from its visible message channels to all.

Agents only output to the channels they can see. This function expands the output to all channels, by filling in fill_value for the logits in the channels the agent cannot see.

Parameters:

• agent_name (str) – The name of the agent.

• output (NDArray) – An output of an agent. This is a single key in the output of the agent’s forward pass.

• shape_spec (str) – The shape of the output. This is a space-separated string of the dimensions of the output. One of these must be “channel”.

• fill_value (Any, default=-1e9) – The value to fill in for the channels the agent cannot see.

Returns:

expanded_output (NDArray) – The output expanded to all channels. This has the same shape as output, except that the channel dimension is the full set of message channels.

_restrict_input_to_visible_channels(agent_name: str, input_array: Tensor | ndarray[Any, dtype[_ScalarType_co]], shape_spec: str) → Tensor

Restrict an agent’s input to its visible message channels.

Agents only receive messages from the channels they can see. This function restricts the input to the agent to only the visible message channels.

Parameters:

• agent_name (str) – The name of the agent.

• input_array (Tensor | NDArray) – The input array to the agent.

• shape_spec (str) – The shape of the input. This is a space-separated string of the dimensions of the input. One of these must be “channel”.

Returns:

restricted_input (Tensor | NDArray) – The input restricted to the visible message channels.

forward(data: NestedArrayDict, environment: PureTextEnvironment) → NestedArrayDict

Run the forward pass through all agent parts and combine the outputs.

Parameters:

• data (NestedArrayDict) –

  The input data. A nested dictionary of arrays with keys:

  • ”round” (…): The current round number.

  • ”seed” (…): The per-environment seed.

  • ”message_history” (… round channel): The history of messages exchanged

    between the agents in each channel.

  • ”message_agent_id” (… round channel): The id of the agent who messaged

    at a round-channel pair.

  • ”question” (…): The problem text.

  • ”solution” (…): The proposed solution text.

  • ”prover_stance” (…): When randomizing the prover stance, the verdict

    that the prover is arguing for, where 0 means “reject” and 1 means “accept”.

• environment (PureTextEnvironment) – The environment in which the agents are running.

Returns:

data (NestedArrayDict) – The input data updated with the output of running the agent. This has the added keys:

• (“agents”, “message”) (… agent channel): The message generated by the

  agent.

• (“agents”, “raw_message”) (… agent): The raw message generated by the

  models, before parsing.

• (“agents”, “prompt”) (… agent message field): The prompt used to

  generate the message.

• (“agents”, “decision”) (… agent): The decision made by the agent. This

  is either 0 (reject), 1 (accept) or 2 (no decision).

• (“agents”, “retry_count”) (… agent channel): The number of retries

  before the generation was successful for each agent.

• (“agents”, “token_limit”) (… agent channel): A boolean array indicating

  whether the generation was stopped because the maximum number of tokens was reached for each agent.