SNOW-1792786: --config-file ... does not work when connection.toml is present

[tylerjt-cvna]

SnowCLI version

3.0.2

Python version

3.12.3

Platform

Windows-10-10.0.19045-SP0

What happened

snow --config-file .\my.config.toml does not work if there is a connections.toml file in the users home directory (I'm on windows so C:\Users<UserName>\.snowflake). This is a problem because the VS Code Snowflake plugin requires the connection.toml file to work. However, our team would like to structure our project directory so that connections are stored in the project rather than a user config file.

Console output

PS C:\Users\User\source\repos\temp> snow --config-file ./prod.config.toml connection list
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| connection_name | parameters                                                                                                                                                                       | is_default |
|-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------|
| prod            | {'account': '<account>', 'user': '<username>', 'database': '<database>', 'schema': '<schema>', 'warehouse': '<warehouse>', 'role': '<role>', 'authenticator': 'externalbrowser'} | False      |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
PS C:\Users\User\source\repos\temp> echo $null >> C:\Users\User\.snowflake\connections.toml
PS C:\Users\User\source\repos\temp> snow --config-file ./prod.config.toml connection list      
Reading connections from C:\Users\User\.snowflake\connections.toml. Entries from config.toml are ignored.
No data

How to reproduce

snow --config-file ./prod.config.toml connection list
echo $null >> C:\Users\<Username>\.snowflake\connections.toml
snow --config-file ./prod.config.toml connection list

[sfc-gh-pjob]

Hello, as you have described, at the moment connections from connections.toml always override connections from config.toml (it doesn't matter if it is a default config or a custom config). The rest of the settings from config.toml should work. I don't consider it as a bug because it's expected behaviour - our configuration management relies on Snowflake's Python Connector which uses connections.toml as a primary source for connections and we do the same.

However I see that it can be a real issue in your case. We have plans to redesign connections configuration management but it will take some time. At the moment, I have only one idea for a workaround but it isn't so nice and simple as I'd like it to be.

IMO you can set SNOWFLAKE_HOME env variable for CLI's shell session to point to any other directory than your directory with connections.toml. Thanks to that, CLI won't find connections.toml. You have to remember that default logs location will change too but you can change the path to logs in config.toml (see logs configuration docs). Different value for SNOWFLAKE_HOME should be probably set only for the shell running CLI to not interfere with other Snowflake's tools/libraries.

I know that this workaround is far away from an elegant solution but I'm afraid it's the only one which I can suggest at the moment. Can you tell me if it can work for you?

I'll post you updated about our plans to change the current behaviour.

[tylerjt-cvna]

Hi - Thanks for getting back. This workaround appears to be working. I am able to use the Snowflake VS Code plugin and our project-embedded .toml file at the same time. This should be fine for our use case.

I appreciate you looking into this issue and converting it to a feature request. It would be very nice to be able to override the default connection.toml file for use cases like this.