Revamp submit train script for evals

[natolambert]

We can now use scripts/submit_finetune_jobs.py with the config structure.
How it works:

1. Loads a beaker config that is mostly filled out
2. Optionally loads a config from the local directory (e.g. in configs/train_configs/)
3. Optionally takes in additional command line arguments

The file overrides from the end. Command line arguments have highest priority, and those over the passed config, and those over the default config.

This allows things like running parameter sweeps via bash script via the following (can even put these in loops)

python scripts/submit_finetune_job.py --config=configs/train_configs/sft/default.yaml  --learning_rate 1e-6
python scripts/submit_finetune_job.py --config=configs/train_configs/sft/default.yaml  --learning_rate 4e-6
python scripts/submit_finetune_job.py --config=configs/train_configs/sft/default.yaml  --learning_rate 1e-5
python scripts/submit_finetune_job.py --config=configs/train_configs/sft/default.yaml  --learning_rate 4e-5

Run with

sh scripts/submit_finetune_jobs.sh

[natolambert]

UPDATE: This is due to the dockerfile! We should maintain a oe-adapt image rather than a hamish one so either to update.

Currently debugging this. Getting a error with the paths (not sure why it's happening / where commands are fun in Beaker jobs)

2024-07-12T22:04:38.702970347Z Traceback (most recent call last):
2024-07-12T22:04:38.703003402Z   File "/stage/open_instruct/finetune.py", line 49, in <module>
2024-07-12T22:04:38.703007590Z     from open_instruct.utils import ArgumentParserPlus, FlatArguments
2024-07-12T22:04:38.703011147Z ModuleNotFoundError: No module named 'open_instruct'
2024-07-12T22:04:38.748507759Z Traceback (most recent call last):
2024-07-12T22:04:38.748545263Z   File "/stage/open_instruct/finetune.py", line 49, in <module>
2024-07-12T22:04:38.748557256Z     from open_instruct.utils import ArgumentParserPlus, FlatArguments
2024-07-12T22:04:38.748561334Z ModuleNotFoundError: No module named 'open_instruct'
2024-07-12T22:04:38.806342385Z Traceback (most recent call last):
2024-07-12T22:04:38.806382674Z   File "/stage/open_instruct/finetune.py", line 49, in <module>
2024-07-12T22:04:38.806388846Z     from open_instruct.utils import ArgumentParserPlus, FlatArguments
2024-07-12T22:04:38.806393084Z ModuleNotFoundError: No module named 'open_instruct'
2024-07-12T22:04:38.874596426Z Traceback (most recent call last):
2024-07-12T22:04:38.874617337Z   File "/stage/open_instruct/finetune.py", line 49, in <module>
2024-07-12T22:04:38.874640672Z     from open_instruct.utils import ArgumentParserPlus, FlatArguments
2024-07-12T22:04:38.874659630Z ModuleNotFoundError: No module named 'open_instruct'

[hamishivi]

this might expose the wandb key in the beaker config, rather than set in the workspace as a secret.... I would prefer we just have a WANDB KEY secret in the workspace, and the workspace owner sets it.

[natolambert]

Yeah let me fix this and add instructions.

[hamishivi]

gonna more or less assume this parsing works. would probably be good to test a bit first.

[natolambert]

I've tested parsing a decent bit. Was making sure all the values passed and from config are right, both for values after --arg value, and --store_true keys. Maybe should make sure it works for list keys (very niche case, e.g. log to wandb and tensorboard).

I have some jobs in the queue for this now.

[natolambert]

(accidentally clicked close)