nip.code_validation.dataset_generation._try_generate_buggy_solutions

nip.code_validation.dataset_generation._try_generate_buggy_solutions#

nip.code_validation.dataset_generation._try_generate_buggy_solutions(datum: dict, model: str, system_prompt: str, max_modifications: int, fraction_to_modify: float = 0.5, max_attempts: int = 10, multiple_completions: bool = True, existing_buggy_solutions: list[dict] | None = None) → list[str | None][source]#

Generate buggy solutions by modifying a fraction of the provided solutions.

Parameters:

  • datum (dict) – The corresponding datum from the original APPS dataset.

  • model (str) – The model to use for generating buggy solutions.

  • system_prompt (str) – The system prompt to provide context to the model.

  • fraction_to_modify (float) – The fraction of solutions to modify.

  • max_modifications (int) – The maximum number of solutions to modify.

  • max_attempts (int, optional) – The maximum number of attempts to generate a valid buggy solution. Defaults to 10.

  • multiple_completions (bool, optional) – If True, generate multiple completions for each prompt. Defaults to True.

  • existing_buggy_solutions (Optional[list[dict]], optional) – A list of existing buggy solutions to update or extend. Defaults to None.

Returns:

list[str | None] – A list of buggy solutions or None if a valid buggy solution could not be generated.