Global config finetuning

[Desmond819]

Hi everyone,

Can we actually update some of the setup on global config to increase the performance (inference speed)? Any idea how each of the parameters is affecting the generation?

Thanks

[hnyls2002]

@Desmond819

1. Sure, to modify the global config, we can expose the modifying API or use environment variables.
2. You can expect the illustration in our detailed document, which is on the way.

[merrymercy]

The fields in global_config.py are not very useful or very advanced. This doc https://github.com/sgl-project/sglang/blob/main/docs/en/hyperparameter_tuning.md should be enough for most cases.