nip.code_validation.dataset_generation._load_cv_dataset

nip.code_validation.dataset_generation._load_cv_dataset#

nip.code_validation.dataset_generation._load_cv_dataset(config: CodeValidationDatasetConfig, splits: list[str]) Dataset[source]#

Load an existing code validation dataset or create an empty one.

First try to load the dataset from the Hugging Face Hub, then try to load it from the local directory. If the dataset is not found, create an empty dataset.

Parameters:
Returns:

buggy_data (datasets.Dataset) – The code validation dataset.

Raises:

ValueError – If neither the Hugging Face Hub repository nor the local directory are specified in the configuration.