cli: `dlt ai IDE setup` command added #2503

zilto · 2025-04-10T00:44:10Z

This new CLI command allows to generate "rules files" for the user's project according to IDE-specific specs. This follows similar logic as dlt init command for core sources.

Design choices

The format dlt ai IDE setup was used, but it could be dlt ai setup IDE

TODO

improve rules files content
support more IDEs
add tests
think about mechanism to "update" all rules files at once

netlify · 2025-04-10T00:44:29Z

✅ Deploy Preview for dlt-hub-docs canceled.

Name	Link
🔨 Latest commit	`172dbbe`
🔍 Latest deploy log	https://app.netlify.com/sites/dlt-hub-docs/deploys/67fec5c19fb9290008dea7a3

rudolfix

command is good and well structured. but at the end @adrianbr needs to give it a test run.

there's one thing I'd change: please re-use the code that clones verified sources and take files from there. we have a place for it
https://github.com/dlt-hub/verified-sources/tree/master/init

right now it is not used (we moved init template to core code) so you can wipe it out and place the files there.

AFAIK @adrianbr also plans to put a few docs files that may be common to all editors. we'll know soon

zilto · 2025-04-10T17:29:21Z

@rudolfix If I understand, you suggest placing rules.md files in dlt-hub/verified-sources/init. This directory is no longer used by dlt, so I could just used the dlt function "get files from verified-sources/init" instead of implementing new logic

Questions:

wouldn't older dlt version still pull from dlt-hub/verified-sources/init? Deleting files would be a breaking change forcing version upgrade
any reason to put this files in dlt-hub/verified-sources vs. dlt-hub/dlt? Easier to update the rules files? The downside I see is that rules should probably match the dlt version (easy to manually fix though)
for code reuse, I looked at dlt pipeline init command it's not really reusable without refactoring. This PR implements the same logic but trimmed

rudolfix · 2025-04-13T16:39:58Z

@zilto my goal here is to let @adrianbr and @AstrakhantsevaAA to quickly iterate the rules without re-releasing our library.

we did that with init repo (discontinued) and then with verified-sources. btw. this is also the reason I used a real running code without any templating and did all the changes by parsing Python to AST and rewriting scripts.. but I digress :)

Answers:

wouldn't older dlt version still pull from dlt-hub/verified-sources/init? Deleting files would be a breaking change forcing version upgrade

version 0.5.x did that and users have already instructions to use --branch 0.5.x (AFAIK) argument if they want to pull compatible version. and what I can see, the old docs are down anyway... so I bet this is not a problem

any reason to put this files in dlt-hub/verified-sources vs. dlt-hub/dlt? Easier to update the rules files? The downside I see is that rules should probably match the dlt version (easy to manually fix though)

the idea was that verified sources is way smaller to clone locally and making PRs is easier. is it really like this? for init scripts it definitely was (there was just simple linting that checked if dlt command will run for all possible init variants)

backward compat: we planned to use version tags to pull always the right version of verified sources. but ultimately such strict biding was never necessary. sill users can pass their own location and branch to init command so they can run their own repos with their private code

for code reuse, I looked at dlt pipeline init command it's not really reusable without refactoring. This PR implements the same logic but trimmed

I meant code that clones the verified source repo into know location. it is in dlt.common.git module. it does a few tricks to minimize the pulls, also you can pass branches and tags to it and it will clone and switch correctly

tldr;> see first sentence of my comment.

rudolfix · 2025-04-14T10:28:15Z

dlt/cli/ai_command.py

+    """
+    specs = IDE_RULES_SPECS[ide]
+
+    src_storage = FileStorage(str(Path(dlt.__file__).parent.joinpath(RULES_TEMPLATE_RELATIVE_PATH)))


I'd go for something way dumber. you have

/ai folder

it has subfolders with names of the editors

those subfolders contains files to copy into run_dir

the command looks if it has a subfolder with editor name and copies all the files

this way you can add and remove editors, add more files etc. without changes in the code. WDYT?

OFC you should not overwrite existing files.

rudolfix · 2025-04-14T10:35:26Z

dlt/cli/ai_command.py

+
+
+def mcp_command() -> None:
+    ...


this will be cool. questions:

should we integrate it into cli that actually runs mcp? for example dlt mcp info would show it?

should support both stdio and sse (with port) to make stuff easier?
a very top thing would be to somehow be aware of which editor is set up and display info for it without asking for a type

rudolfix · 2025-04-14T10:36:27Z

dlt/cli/plugins.py

+        # ai_mcp_cmd = ai_subparsers.add_parser("mcp", help="Launch the dlt MCP server")
+
+    def execute(self, args: argparse.Namespace) -> None:
+        ai_setup_command_wrapper(args.ide)


should we check if project is initialized already? ie. if .dlt folder exists (you check if run_context.settings exist)

zilto · 2025-04-15T00:48:01Z

See this PR dlt-hub/verified-sources#608 for related files

rudolfix

LGTM! is there a branch with the content in verified sources already?

rudolfix

this is really good. we still miss documentation.

actually command line reference autogenerates from argparse so we may put more info there
maybe we could tell people how to add or fix the rules using verified sources?

dlt ai setup command added

150d850

zilto self-assigned this Apr 10, 2025

zilto requested a review from sh-rp April 10, 2025 00:44

rudolfix requested review from rudolfix and removed request for sh-rp April 10, 2025 09:39

rudolfix requested changes Apr 10, 2025

View reviewed changes

rudolfix reviewed Apr 14, 2025

View reviewed changes

load files from dlt-hub/verified-sources

96b3a58

zilto mentioned this pull request Apr 15, 2025

feat: add reference LLM-enabled IDE setup files dlt-hub/verified-sources#608

Open

zilto requested a review from rudolfix April 15, 2025 00:46

remove files from main repo

76700f4

rudolfix reviewed Apr 15, 2025

View reviewed changes

linting

172dbbe

rudolfix reviewed Apr 16, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cli: `dlt ai IDE setup` command added #2503

cli: `dlt ai IDE setup` command added #2503

zilto commented Apr 10, 2025

netlify bot commented Apr 10, 2025 •

edited

Loading

rudolfix left a comment

zilto commented Apr 10, 2025

rudolfix commented Apr 13, 2025 •

edited

Loading

rudolfix Apr 14, 2025

rudolfix Apr 14, 2025

rudolfix Apr 14, 2025

zilto commented Apr 15, 2025

rudolfix left a comment

rudolfix left a comment •

edited

Loading

cli: dlt ai IDE setup command added #2503

Are you sure you want to change the base?

cli: dlt ai IDE setup command added #2503

Conversation

zilto commented Apr 10, 2025

Design choices

TODO

netlify bot commented Apr 10, 2025 • edited Loading

✅ Deploy Preview for dlt-hub-docs canceled.

rudolfix left a comment

Choose a reason for hiding this comment

zilto commented Apr 10, 2025

rudolfix commented Apr 13, 2025 • edited Loading

rudolfix Apr 14, 2025

Choose a reason for hiding this comment

rudolfix Apr 14, 2025

Choose a reason for hiding this comment

rudolfix Apr 14, 2025

Choose a reason for hiding this comment

zilto commented Apr 15, 2025

rudolfix left a comment

Choose a reason for hiding this comment

rudolfix left a comment • edited Loading

Choose a reason for hiding this comment

cli: `dlt ai IDE setup` command added #2503

cli: `dlt ai IDE setup` command added #2503

netlify bot commented Apr 10, 2025 •

edited

Loading

rudolfix commented Apr 13, 2025 •

edited

Loading

rudolfix left a comment •

edited

Loading