Experiment Scripts

The following scripts are available in the scripts directory. These scripts include ones for running experiments with a grid of hyper-parameters, as well as scripts for creating datasets and various other tasks.

The Graph Isomorphism Task

ppo_gi	PPO graph isomorphism experiment using a grid of hyperparameters.
dummy_test_gi	Dummy graph isomorphism test script, for debugging purposes.
solo_agents_gi	Test solo graph isomorphism agents using a grid of hyperparameters.
create_gi_dataset	Generate a graph isomorphism dataset.

The Image Classification Task

ppo_ic	PPO image classification experiment using a grid of hyperparameters.
solo_agents_ic	Test solo image classification agents using a grid of hyperparameters.
dummy_test_ic	Dummy image classification test script, for debugging purposes.

The Code Validation Task

cv_experiment	Script for running code validation experiments.
create_cv_dataset	Generate a Buggy APPS dataset.
download_cv_checkpoints	Download code validation checkpoints from W&B to the local directory.
visualise_malt_rollouts	Visualise the rollouts of a MALT experiment as a forest of trees.

Language Model Server

run_lm_server

Run the self-hosting language model server.

Miscellaneous Scripts

download_all_datasets	Download all datasets used in the project.
time_timeable	Script to time a timeable action.