nip.utils.experiments.HyperparameterExperiment

class nip.utils.experiments.HyperparameterExperiment(param_grid: dict, experiment_fn: Callable[[ExperimentFunctionArguments], None], run_id_fn: Callable[[int | None, Namespace], str] | None = None, experiment_name: str = 'EXPERIMENT', run_preparer_fn: Callable[[dict, Namespace], PreparedExperimentInfo] | None = None, arg_parser_description: str = 'Run hyperparameter experiments', default_wandb_project: str | None = None, allow_resuming_wandb_run: bool = False, add_run_infix_argument: bool = True)

A base class to run an experiment over a grid of hyperparameters.

Runs each combination of hyperparameters in the grid as a separate experiment.

Methods Summary

__init__(param_grid, experiment_fn[, ...])
_run(base_logger)	Run the experiment.
check_no_extant_runs()	Make sure there are no runs with the same ID as any run in this experiment.
run()	Run the experiment.

Attributes

combinations	An iterator over the combinations of hyperparameters.
common_run_name	A name for the experiment that is common to all runs.
enumerated_combinations	An iterator over the combinations of hyperparameters plus an enumeration.

Methods

__init__(param_grid: dict, experiment_fn: Callable[[ExperimentFunctionArguments], None], run_id_fn: Callable[[int | None, Namespace], str] | None = None, experiment_name: str = 'EXPERIMENT', run_preparer_fn: Callable[[dict, Namespace], PreparedExperimentInfo] | None = None, arg_parser_description: str = 'Run hyperparameter experiments', default_wandb_project: str | None = None, allow_resuming_wandb_run: bool = False, add_run_infix_argument: bool = True)

abstract _run(base_logger: Logger)

Run the experiment.

This is the function that actually runs the experiment, and should be implemented by subclasses.

check_no_extant_runs()

Make sure there are no runs with the same ID as any run in this experiment.

Raises:

ValueError – If there is a run with the same ID as any run in this experiment.

run()

Run the experiment.