nip.code_validation.rollout_analysis#

Use a language model to do analysis on rollouts.

Classes

`BinaryRolloutAnalyser`(hyper_params, ...[, ...])	Base class for rollout analyser which yield a binary classification.
`CodeValidationRolloutAnalyser`(hyper_params, ...)	Base class for analysing code validation rollouts.
`FollowUpAnalyser`(hyper_params, settings, ...)	Analyse whether the prover quotes accurately from the solution.
`ProverAccurateQuotationAnalyser`(...[, ...])	Analyse whether the prover quotes accurately from the solution.
`ProverAnalyserMixin`()	Mixin class for analysing provers.
`ProverDirectQuotationAnalyser`(hyper_params, ...)	Analyse when the prover directly quotes the solution.
`ProverRoleConformanceAnalyser`(hyper_params, ...)	A watchdog to evaluate how well the prover(s) are conforming to their roles.