nip.parameters.scenario.CodeValidationParameters

class nip.parameters.scenario.CodeValidationParameters(apps_difficulty: Literal['introductory', 'interview', 'competition'] = 'interview', apps_solution_number: int = 0, apps_exclude_flagged: bool = True)

Additional parameters for the code validation task.

Parameters:

• app_difficulty (Literal["introductory", "interview", "competition"]) – The difficulty level of the APPS dataset, if using.

• app_solution_number (int) – Each question in the APPS dataset has multiple solutions. This parameter specifies which solution to use.

• apps_exclude_flagged (bool) – All datapoints in the Buggy APPS dataset have been passed through the OpenAI moderation API, with any flags noted. If this parameter is set to True, flagged datapoints will be excluded from the dataset.

Methods Summary

__eq__(other)	Return self==value.
__init__([apps_difficulty, ...])
__post_init__()
__repr__()	Return repr(self).
_get_param_class_from_dict(param_dict)	Try to get the parameter class from a dictionary of serialised parameters.
construct_test_params()	Construct a set of basic parameters for testing.
from_dict(params_dict[, ignore_extra_keys])	Create a parameters object from a dictionary.
get(address)	Get a value from the parameters object using a dot-separated address.
to_dict()	Convert the parameters object to a dictionary.

Attributes

apps_difficulty
apps_exclude_flagged
apps_solution_number

Methods

__eq__(other)

Return self==value.

__init__(apps_difficulty: Literal['introductory', 'interview', 'competition'] = 'interview', apps_solution_number: int = 0, apps_exclude_flagged: bool = True) → None

__post_init__()

__repr__()

Return repr(self).

classmethod _get_param_class_from_dict(param_dict: dict) → type[ParameterValue] | None

Try to get the parameter class from a dictionary of serialised parameters.

Parameters:

param_dict (dict) – A dictionary of parameters, which may have come from a to_dict method. This dictionary may contain a _type key, which is used to determine the class of the parameter.

Returns:

param_class (type[ParameterValue] | None) – The class of the parameter, if it can be determined.

Raises:

ValueError – If the class specified in the dictionary is not a valid parameter class.

classmethod construct_test_params() → BaseHyperParameters

Construct a set of basic parameters for testing.

classmethod from_dict(params_dict: dict, ignore_extra_keys: bool = False) → BaseHyperParameters

Create a parameters object from a dictionary.

Parameters:

• params_dict (dict) – A dictionary of the parameters.

• ignore_extra_keys (bool, default=False) – If True, ignore keys in the dictionary that do not correspond to fields in the parameters object.

Returns:

hyper_params (BaseParameters) – The parameters object.

get(address: str) → Any

Get a value from the parameters object using a dot-separated address.

Parameters:

address (str) – The path to the value in the parameters object, separated by dots.

Returns:

value (Any) – The value at the address.

Raises:

KeyError – If the address does not exist.

to_dict() → dict

Convert the parameters object to a dictionary.

Turns enums into strings, and sub-parameters into dictionaries. Includes the is_random parameter if it exists.

Returns:

params_dict (dict) – A dictionary of the parameters.