How does Python LSP integration work?

[cbarrete]

My understanding is that Python LSP integration is supposed to be built using the source-db subtargets (and accompanying BXL scripts in the prelude), but I cannot find any mentions of "source db" in language server docs.
I'm very unfamiliar with the Python ecosystem, could somebody give me some pointers for where to start looking?

My understanding is that in the Bazel world, this is achieved by making venvs that contain both third-party and local dependencies. Is it similar in the Buck2 world?

I've looked at the git log for prelude/python/sourcedb/ide.bxl, but it's not quite enough context I believe.

[connernilsen]

Hey @cbarrete, I have an understanding of how this works internally for the Pyre type checker, but let me see if I understand your question correctly. Are you looking to build the source-db and use it in some other language server? Which Python LSP integration do you mean?

The source-db is a mapping from files on disk to where files would end up in a par (so essentially how files get remapped in the resulting executable). It also makes sure any other generated files are 'ensured', so they exist on disk for you to utilize. Depending on what you're looking for here, one of the other sourcedb scripts might be more useful to you, or this may be doing something other than what you're expecting.

For Pyre, we use the resulting sourcedb to understand what file an import in a Python file actually points to, and to make sure something like a .pyi file generated from specific c(++) rules exist for us to use.

[cbarrete]

Hey, sorry for the delay and thanks for your reply!

What I want is to setup a good Python dev experience, including:

• formatting
• linting (unused variables, common mistakes, etc.)
• type checking
• code navigation
• refactoring

There are various tools and language servers that can do this, but most of them require a "proper" Python setup, which typically means a pyproject.toml, which I obviously don't want to have to maintain given that I use Buck.

Ruff can handle some parts of this without much configuration, but e.g. type checking ore more advanced linting require project metadata. My understanding is that the prelude exposes this via the source-db file (which also depends dependencies so that they get materialized, like you mentioned). However, most language servers (e.g. https://github.com/python-lsp/python-lsp-server) don't seem to know about this: as I understand it, this is not a standard file in the Python ecosystem.

I'm fine with using Pyre (it's not ideal to be locked into a single type checker, but oh well), but the docs for how to integrate it into an editor are missing. The configuration docs mention some JSON config, but the schema doesn't match that of a source-db output. Building a source-db file (either via a target's subtarget or using BXL) is no problem, but I don't know what to do with it!

Is there some documentation somewhere about how to get Pyre (or ideally any language server) to use a sourcedb?

[cormacrelf]

There are examples for using LSP (pyright) in here: https://github.com/cormacrelf/elk

Specifically https://github.com/cormacrelf/elk/blob/master/example/venv.sh. Just launch your LSP with this as the wrapper. You can probably use VSCode's pylance this way to get a full dev experience. None of this requires using elk, actually. You can drop this in any old buck project.

(I think elk would be better rewritten with uv instead of poetry, but it does go. The tricky part is specifying the tags in your elk.toml: manylinux2014, all that cactus python packaging hackery.)