Allow CLI overrides of project.yml

[BramVanroy]

You can overwrite options of the training config file, like so

spacy train config.cfg --paths.train ./corpus/train.spacy --paths.dev ./corpus/dev.spacy --training.batch_size 128

This is useful in its own right but it would be very helpful if the same can be applied to the project.yml file. That would make a lot of sense if you want to train a model with the same pipeline and data, but different configs. You could then even overwrite config values in one go.

Take for example this (incomplete) project.yml.

vars:
  config: "ud_trf"
  lang: "en"
  batch_size: 64

commands:
  - name: train
    help: "Train Transformer pipeline"
    script:
      - "python -m spacy train configs/${vars.config}.cfg --nlp.lang ${vars.lang} --training.batch_size ${vars.batch_size}

As it is implemented now, running spacy project run train will always interpolate to

python -m spacy train configs/ud_trf.cfg --nlp.lang en --training.batch_size 64

But it would be very useful to be have a dynamic project.yml file as well which can pass its dynamic values to the commands. That allows us to do something like:

spacy project --vars.config ud_sm --vars.batch_size 1024 run train

If batch_size was not a var in project.yml but only present in the config file under training.batch_size`, you would then be able to do something like

spacy project --vars.config ud_sm run train --training.batch_size 1024 

So overwriting values in project.yml as well as in the related config file.

The major advantage is that you only need one project file with rather static components in the workflow (e.g. data preprocessing) and other dynamic ones (training and packaging). It is a very DRY approach of project management - you do not need to repeat anything across different clones of a project. Particularly in terms of data/assets this is a plus.

My use case is that I am training different models on the same data, both in terms of architecture and hyperparameters. I could call data preparation scripts separately and then run spacy train for the separate models, but that defeats the purpose of having a project file in the first place. In other words: I still want to use the workflows in the project.yml file, but more dynamically.

[svlandeg]

Hm, have you actually tried this?

spacy project --vars.config ud_sm --vars.batch_size 1024 run train

or perhaps

spacy project run train --vars.config ud_sm --vars.batch_size 1024 

I thought that should work?

cf. a previous discussion & issue reported here: #7734

[BramVanroy]

I did try both before posting. Neither work for me. I used a config similar to the one above, which is a modification of this one. (Also tried running it from Python command with python -m to no avail.)

> spacy project --vars.config .\configs\en_ud_sm.cfg run train
Usage: python -m spacy project [OPTIONS] COMMAND [ARGS]...
Try 'python -m spacy project --help' for help.

> spacy project run train --vars.config .\configs\en_ud_sm.cfg
Usage: python -m spacy project run [OPTIONS] [SUBCOMMAND] [PROJECT_DIR]

Error: Invalid value for '[PROJECT_DIR]': Directory '--vars.config' does not exist.

============================== Info about spaCy ==============================

spaCy version 3.1.0
Location C:\Users\bramv.virtualenvs\spacy_ud-xCJQKBA9\lib\site-packages\spacy
Platform Windows-10-10.0.19041-SP0
Python version 3.7.9
Pipelines en_core_web_sm (3.1.0), en_core_web_trf (3.1.0), en_ud_en_ewt_trf (1.0.0)

[svlandeg]

Right, can you add a . for the PROJECT_DIR and try again?

spacy project run train . --vars.config ud_sm --vars.batch_size 1024