nip.utils.plotting.decision_spectrum.get_decision_histogram

nip.utils.plotting.decision_spectrum.get_decision_histogram(rollouts: NestedArrayDict, hyper_params: HyperParameters, bins: int | str = 'auto', include_no_decision: Literal['yes', 'no', 'if_nonzero'] = 'no') → tuple[ndarray[Any, dtype[_ScalarType_co]], list[str]]

Compute a histogram of the verifier decisions.

Depending on the parameters of the experiment, the verifier can make a variety of decisions. This function computes a histogram of these decisions.

Parameters:

• rollouts (NestedArrayDict) – The rollouts to be analysed. Each rollout is a NestedArrayDict containing the verifier decisions.

• hyper_params (HyperParameters) – The hyperparameters of the experiment. This is used to determine the decision spectrum used by the verifier.

• bins (int | str, default="auto") – The number of bins to use for the histogram. This is passed to the numpy.histogram() function. See the documentation for more details.

• include_no_decision (Literal["yes", "no", "if_nonzero"], default="no") – If “yes”, the histogram includes the “no_decision” option, for when the verifier does not make a decision by the end of the rollout. This is added at the end of the histogram with label “no_decision”. If “no”, these rollouts are ignored. If “if_nonzero”, the histogram includes the “no_decision” option only if there are any rollouts with no decisions.

Returns:

• histogram (NDArray) – The histogram of the verifier decisions. This is a 1D array of shape (n_bins,).

• bin_labels (list[str]) – The labels for the bins. This is a list of strings, where each string is the label for the corresponding bin in the histogram. If multiple values are grouped into a single bin, the labels are of the form “{first_value}-{last_value}”. This is a list of length n_bins.