Experiment Scripts#

The following scripts are available in the scripts directory. These scripts include ones for running experiments with a grid of hyper-parameters, as well as scripts for creating datasets and various other tasks.

The Graph Isomorphism Task#

ppo_gi

PPO graph isomorphism experiment using a grid of hyperparameters.

dummy_test_gi

Dummy graph isomorphism test script, for debugging purposes.

solo_agents_gi

Test solo graph isomorphism agents using a grid of hyperparameters.

create_gi_dataset

Generate a graph isomorphism dataset.

The Image Classification Task#

ppo_ic

PPO image classification experiment using a grid of hyperparameters.

solo_agents_ic

Test solo image classification agents using a grid of hyperparameters.

dummy_test_ic

Dummy image classification test script, for debugging purposes.

The Code Validation Task#

cv_experiment

Script for running code validation experiments.

create_cv_dataset

Generate a Buggy APPS dataset.

download_cv_checkpoints

Download code validation checkpoints from W&B to the local directory.

visualise_malt_rollouts

Visualise the rollouts of a MALT experiment as a forest of trees.

Language Model Server#

run_lm_server

Run the self-hosting language model server.

Miscellaneous Scripts#

download_all_datasets

Download all datasets used in the project.

time_timeable

Script to time a timeable action.