nip.code_validation.dataset_generation._load_cv_dataset

nip.code_validation.dataset_generation._load_cv_dataset(config: CodeValidationDatasetConfig, splits: list[str]) → Dataset

Load an existing code validation dataset or create an empty one.

First try to load the dataset from the Hugging Face Hub, then try to load it from the local directory. If the dataset is not found, create an empty dataset.

Parameters:

• config (CodeValidationDatasetConfig) – The configuration object for the code validation dataset.

• splits (list[str]) – The list of splits to load.

Returns:

buggy_data (datasets.Dataset) – The code validation dataset.

Raises:

ValueError – If neither the Hugging Face Hub repository nor the local directory are specified in the configuration.