How about adding .pyi file support to work with Cython compiled .so/.pyd modules?

[ccll]

Recently I've been fiddling with Cython compilation to protected source code, one problem left is PyInstaller can not analyze compiled .so/.pyd to get imported dependencies, so we need to manually specify those direct dependencis with --hiddenimport.

I also noticed that mypy have a tool stubgen which can generate .pyi file from .py source code, the .pyi files only include the import and type declarations.

Example:

# db.pyi
import asyncpg

db_pool: asyncpg.pool.Pool

def get_db_pool() -> asyncpg.pool.Pool: ...
async def init_db_pool() -> None: ...
async def shutdown_db_pool() -> None: ...

It just like C/C++ header files with forward declarations, which is perfect for PyInstaller to do analysis, we could get a perfect workflow to protect source code with Cython:

1. .py -> cython -> .so/.pyd
2. .py -> stubgen -> .pyi
3. .pyi + .so/.pyd -> PyInstaller -> bundle

EDIT: I just found out that PyInstaller can actually specify multiple entry scripts and '.pyi' file is also supported, so this basic workflow just works! Wooo Hooo!

We could go one step further, that PyInstaller directly integrate Cython and stubgen to automatically pack compiled bundles?
I don't know, but it looks very promising.

In this area Nuitka is also a great tool, but it requires compiling ALL the dependencies to machine code, in real life a small-to-medium FastAPI project could easily cost 30+ minutes to build (even with cache). PyInstaller have the flexibility to selectively compile only main program and keep external dependencies in Python form, where we can get the best of both worlds: fast packing + protected source code.

[bwoodsend]

We could go one step further, that PyInstaller directly integrate Cython and stubgen to automatically pack compiled bundles?

I really don't want to support or endorse this. If you think that the dependency finding is the hard part of using Cython with PyInstaller then you're probably only thinking that because you're blissfully unaware of the woes of ABI compatibility (which PyInstaller can do nothing to help you with).

The last obfuscation feature we had misled a lot of users into doing things that are impossible to do like hiding API keys in an application. To that end, I don't think it's safe for PyInstaller to have any built in obfuscation support.

[rokm]

As of #9084 the analysis is supposed to pick up and analyze .pyi (or .py) files found next to extension modules. If that is not happening in your case, we might need to investigate as to why.

While I suppose we could look into automatic cythonization as an extension of the module_collection_mode mechanism, I would strongly prefer that you tested your cythonized (or otherwise obfuscated) modules before trying to pack them with PyInstaller.