nip.code_validation.dataset_generation._try_generate_buggy_solutions#
- nip.code_validation.dataset_generation._try_generate_buggy_solutions(datum: dict, model: str, system_prompt: str, max_modifications: int, fraction_to_modify: float = 0.5, max_attempts: int = 10, multiple_completions: bool = True, existing_buggy_solutions: list[dict] | None = None) list[str | None] [source]#
Generate buggy solutions by modifying a fraction of the provided solutions.
- Parameters:
datum (dict) – The corresponding datum from the original APPS dataset.
model (str) – The model to use for generating buggy solutions.
system_prompt (str) – The system prompt to provide context to the model.
fraction_to_modify (float) – The fraction of solutions to modify.
max_modifications (int) – The maximum number of solutions to modify.
max_attempts (int, optional) – The maximum number of attempts to generate a valid buggy solution. Defaults to 10.
multiple_completions (bool, optional) – If True, generate multiple completions for each prompt. Defaults to True.
existing_buggy_solutions (Optional[list[dict]], optional) – A list of existing buggy solutions to update or extend. Defaults to None.
- Returns:
list[str | None] – A list of buggy solutions or None if a valid buggy solution could not be generated.