nip.code_validation.data.BuggyAppsCodeValidationDataset

class nip.code_validation.data.BuggyAppsCodeValidationDataset(hyper_params: HyperParameters, settings: ExperimentSettings, protocol_handler: ProtocolHandler, train: bool = True)

An extension of the APPS [HBK+21] dataset with buggy solutions.

Buggy solutions were generated by asking GPT-4o to introduce bugs into the non-buggy solutions from the APPS dataset.

Methods Summary

__getitem__(index)
__init__(hyper_params, settings, ...[, train])
__len__()
__repr__()	Return repr(self).
_load_raw_dataset()	Load the dataset.
_process_data(raw_dataset)	Process the dataset.

Attributes

dataset_filepath_name	The name of the dataset file.
instance_keys	The keys specifying the input instance.
keys	The keys (field names) in the dataset.
processed_dir	The path to the directory containing the processed data.
raw_dir	The path to the directory containing the raw data.

Methods

__getitem__(index: Any) → NestedArrayDict

__init__(hyper_params: HyperParameters, settings: ExperimentSettings, protocol_handler: ProtocolHandler, train: bool = True)

__len__() → int

__repr__() → str

Return repr(self).

_load_raw_dataset() → Dataset

Load the dataset.

Returns:

raw_data (HuggingFaceDataset) – The unprocessed dataset.

_process_data(raw_dataset: Dataset) → Dataset

Process the dataset.

Parameters:

raw_dataset (HuggingFaceDataset) – The unprocessed dataset.

Returns:

processed_dataset (HuggingFaceDataset) – The processed dataset.