Add comprehensive style guide for cuda/core/experimental #1316

Andy-Jost · 2025-12-04T20:46:46Z

Summary

This PR adds a comprehensive style guide for Python and Cython code in cuda/core/experimental. The guide establishes clear conventions and best practices to maintain code quality and consistency across the codebase.

What's Included

The style guide covers 18 major areas:

File Structure - Ordering of elements (SPDX headers, imports, __all__, classes, functions)
Package Layout - Organization of .pyx, .pxd, and .py files, subpackage patterns
Import Statements - Five-group organization with alphabetical sorting
Class and Function Definitions - Ordering within classes (dunder methods, methods, properties)
Naming Conventions - PascalCase, snake_case, UPPER_SNAKE_CASE patterns
Type Annotations - Modern PEP 604 union syntax (X | Y | None), forward references
Docstrings - NumPy docstring style with comprehensive examples
Errors and Warnings - Custom exceptions, error messages, warning patterns
Memory Management - Resource lifecycle and cleanup patterns
Thread Safety and Concurrency - Locks, thread-local storage patterns
Cython-Specific Features - cdef/cpdef/def, nogil, inline functions
Constants and Magic Numbers - Naming conventions, CUDA constants
Comments and Inline Documentation - TODO, NOTE patterns
Code Organization Within Files - Structure and helper function placement
Performance Considerations - GIL management, C types, optimization guidelines
API Design Principles - Public vs private API, backward compatibility
CUDA-Specific Patterns - GIL management for CUDA driver API calls
Copyright and Licensing - SPDX format requirements

Key Principles

PEP 8 Base: Follows PEP 8 as the normative base, highlighting project-specific rules
Modern Python: Promotes PEP 604 (X | Y union syntax) and PEP 563 (from __future__ import annotations)
Codebase-Aligned: Examples reflect actual patterns from the codebase
Forward-Looking: Documents current patterns while promoting migration to modern best practices

Benefits

Consistency: Establishes clear conventions for all contributors
Onboarding: Helps new contributors understand codebase patterns quickly
Code Review: Provides objective standards for reviewing code
Maintainability: Reduces cognitive load by standardizing structure and style

File Added

cuda/core/style-guide.md (1,818 lines)

Next Steps

After merge, this guide can be:

Referenced in code reviews
Used as a checklist for new code
Gradually applied to existing code during refactoring
Extended with additional patterns as needed

copy-pr-bot · 2025-12-04T20:46:50Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

Andy-Jost · 2025-12-04T20:52:50Z

Link to rendered document: style-guide.md

Andy-Jost · 2025-12-04T20:55:21Z

I'm interested in feedback on this idea. With a comprehensive style guide, aside from the other benefits listed in the description, we can use LLMs to help refactor, clean, and document the code. If the team is comfortable with this approach, I can begin to apply rules conservatively to the code base.

This commit adds a complete style guide covering conventions for Python and Cython code in cuda/core/experimental. The guide includes: - File structure and organization (SPDX headers, imports, __all__, ordering) - Package layout (.pyx/.pxd/.py files, subpackage patterns) - Import statement organization (5 groups with alphabetical sorting) - Class and function definition ordering (dunder methods, methods, properties) - Naming conventions (PascalCase, snake_case, UPPER_SNAKE_CASE) - Type annotations (PEP 604 union syntax, forward references) - Docstrings (NumPy style with comprehensive examples) - Error handling and warnings (custom exceptions, stacklevel, one-time warnings) - Memory management (resource lifecycle, cleanup patterns) - Thread safety and concurrency (locks, thread-local storage) - Cython-specific features (cdef/cpdef/def, nogil, inline functions) - Constants and magic numbers (naming, CUDA constants) - Comments and inline documentation (TODO, NOTE patterns) - Code organization within files - Performance considerations (GIL management, C types) - API design principles (public vs private, backward compatibility) - CUDA-specific patterns (GIL management for driver API calls) - Copyright and licensing (SPDX format) The guide follows PEP 8 as the base and promotes modern Python practices (PEP 604, PEP 563) while documenting current codebase patterns.

rwgk · 2025-12-05T00:11:06Z

Will look more closely asap.

Drive-by comment: I was surprised to see the new file under cuda_core/cuda/core/. I'd expect it to be under cuda_core/, like other meta-kind-of files, e.g. LICENSE, README.md, etc.

Andy-Jost · 2025-12-05T00:27:11Z

Will look more closely asap.

Drive-by comment: I was surprised to see the new file under cuda_core/cuda/core/. I'd expect it to be under cuda_core/, like other meta-kind-of files, e.g. LICENSE, README.md, etc.

I thought about that, too. I agree it belongs at the root.

Document the two-phase development approach: - Phase 1: Start with Python driver implementation and tests - Phase 2: Optimize by switching to cydriver with nogil blocks Includes step-by-step conversion guide and before/after examples.

rparolin · 2025-12-05T16:24:04Z

Will look more closely asap.
Drive-by comment: I was surprised to see the new file under cuda_core/cuda/core/. I'd expect it to be under cuda_core/, like other meta-kind-of files, e.g. LICENSE, README.md, etc.

I thought about that, too. I agree it belongs at the root.

Do we want this in the root? I'd expect it within a docs folder. That shouldn't impact an LLMs ability to discover and utilize it.

cpcloud

These aren't blocking, but I have a few concerns:

I am a bit concerned about the team all using various agents with this an getting a variety of different results, with no way to actually enforce any of these guidelines without the human effort of checking the guide, and comparing a give changeset to it.
Given the extensive guidelines, I think this will actually add human time for: 1) waiting for the LLM to take action, 2) babysitting it when it, as expected, doesn't follow the guidelines in an acceptable-to-the-author way.

cpcloud · 2025-12-04T21:54:21Z

cuda_core/style-guide.md

+
+Files in `cuda/core/experimental` must follow a consistent structure. The ordering of elements within a file is as follows:
+
+### 1. SPDX Copyright Header


I'm -1 on including things in this guide that can be automated, and I realize that results in a lack of completeness.

In fact, I'd rather our lint rule for this particular thing just add it to the top of the file when it fails.

I think this can be done for all file types, including pyx files.

cpcloud · 2025-12-04T21:56:16Z

cuda_core/style-guide.md

+
+### 3. `__all__` Declaration
+
+If the module exports public API elements, include an `__all__` list after the imports and before any other code. This explicitly defines the public API of the module.


This is slightly misleading, since __all__ is about what happens when you star-import a module.

+1 on defining this, but stating it defines the public API I would remove from the description.

cpcloud · 2025-12-04T21:58:36Z

cuda_core/style-guide.md

+LEGACY_DEFAULT_STREAM = C_LEGACY_DEFAULT_STREAM
+```
+
+### 5. Principal Class or Function


I'm a bit mixed on enforcing ordering of elements here. I do think it can help cut review time, but in practice I find that this kind of strict ordering only starts mattering when files get huge, to which I would probably say "Split the file up" 😆

cpcloud · 2025-12-04T22:01:44Z

cuda_core/style-guide.md

+- Not every file will have all sections. For example, a utility module may not have a principal class.
+- The distinction between "principal" and "other" classes is based on the file's primary purpose. If a file exists primarily to define one class, that class is the principal class.
+- Private implementation functions should be placed at the end of the file to keep the public API visible at the top.
+- **Within each section**, classes and functions should be sorted alphabetically by name. The principal class or function is an exception to this rule, as it appears first in its respective section.


I'm not sure this should be enforced without a linter. It feels a bit draconian to me to require someone to reorder their methods.

For example, I like to order methods by "relevance radius", which is a squishy concept that is something like "methods that are related to each other are close to each other". I find this makes debugging way easier, if only because I can see more related code in a single vim window.

cpcloud · 2025-12-04T22:09:47Z

cuda_core/style-guide.md

+
+2. **Keep `.pxd` files minimal**: Only include declarations needed for Cython compilation. Omit implementation details, docstrings, and Python-only code.
+
+3. **Use `__all__` in submodules**: Each submodule should define `__all__` to explicitly declare its public API.


The public API bit seems to be repeated in a bunch of places, and it's misleading. That said, I'm unsure of a better way to phrase it, but perhaps there's a slightly less "sure" way of stating it?

How about just "Each submodule should define __all__"?

cpcloud · 2025-12-05T17:12:37Z

cuda_core/style-guide.md

+
+### Implementation Comments
+
+Add comments to explain complex logic or non-obvious behavior:


I really question whether this is useful for humans. I suppose it may be useful for an LLM 🤷🏻

cpcloud · 2025-12-05T17:13:09Z

cuda_core/style-guide.md

+
+### Prefer `cdef` for Internal Functions
+
+Use `cdef` functions for internal operations that don't need to be callable from Python:


Isn't this already specified above?

cpcloud · 2025-12-05T17:15:11Z

cuda_core/style-guide.md

+result = []
+for i in range(n):
+    result.append(i)
+
+# Preferred: Use C array or pre-allocate
+cdef int* c_result = <int*>malloc(n * sizeof(int))


Seeing as how these are not equivalent, it seems odd to say "prefer X over Y" when X and Y are very different. Sometimes you want a list, sometimes you don't need it and can use a C array.

cpcloud · 2025-12-05T17:17:06Z

cuda_core/style-guide.md

+Use naming conventions to distinguish public and private APIs:
+
+- **Public API**: No leading underscore, documented in docstrings, included in `__all__`
+- **Private API**: Leading underscore (`_`), may have minimal documentation, not in `__all__`


Since __all__ only affects star import behavior, this is redundant, since leading underscore names are not included in star imports.

cpcloud · 2025-12-05T17:19:31Z

cuda_core/style-guide.md

+
+### Guidelines
+
+1. **Placement**: The copyright header must be the first lines of the file, before any imports or other code.


Isn't this covered by the "all files must include a copyright header"?

mdboom

There is a defacto standard for where to put this kind of stuff and tell the IDE where and when to apply it.

mdboom · 2025-12-08T14:35:10Z

cuda_core/style-guide.md

+- **Abstract base classes**: ABCs that define interfaces (e.g., `MemoryResource` in `_buffer.pyx`)
+- **Other public classes**: Additional classes exported by the module
+
+**All classes and functions in this section should be sorted alphabetically by name**, regardless of their relationship to the principal class. The principal class appears first as an exception to this rule.


-1 on alphabetical ordering. I think we should reflect the logical relationships between classes instead (which I understand is a bit mushy as a concept). It's easy enough in an IDE to jump to the one you are looking for using search.

mdboom · 2025-12-08T14:37:14Z

cuda_core/style-guide.md

+
+- Functions with names starting with `_` (private)
+- `cdef inline` functions used for internal implementation
+- Helper functions not part of the public API


I suppose this is fine, but seems backward to me, given that languages like C have always required forward declarations to do things like this, code tends to be ordered from low-level to high-level, with the "principal public thing" at the bottom, not the top. I realize Python has never had this problem, but it does seems reversed from almost anything else.

mdboom · 2025-12-08T14:39:47Z

cuda_core/style-guide.md

+The `cuda/core/experimental` package uses three types of files:
+
+1. **`.pyx` files**: Cython implementation files containing the actual code
+2. **`.pxd` files**: Cython declaration files containing type definitions and function signatures for C-level access


(For the future): We really should be requiring that all .pyx files have a corresponding .pyi file. Without it, modern IDEs (which don't actually import modules) can not perform auto completion and cross referencing. But we can add that later as an effort to improve the IDE experience.

In 2025, is there a way we can auto-generate .pyi from .py/.pxd/.pyx? If not, let's not make it a must.

One thing I hate the most is Python typing. Does not make any sense to me to maintain 2-3 files.

mdboom · 2025-12-08T14:40:41Z

cuda_core/style-guide.md

+For each `.pyx` file that defines classes or functions used by other Cython modules, create a corresponding `.pxd` file:
+
+- **`.pxd` file**: Contains `cdef` class declarations, `cdef`/`cpdef` function signatures, and `cdef` attribute declarations
+- **`.pyx` file**: Contains the full implementation including Python methods, docstrings, and implementation details


(As above): The better way is to include docstrings and type annotations in a .pyi file and just implementation in the .pyx file.

I don't think Cython would honor .pyi files? To the very least, I don't think it is friendly to write .pyi files when the types are needed to complete a cdef function?

mdboom · 2025-12-08T14:45:28Z

cuda_core/style-guide.md

+
+6. **Separate concerns**: Use `.py` files for pure Python utilities, `.pyx` files for Cython implementations that need C-level performance.
+
+## Import Statements


If I were writing this for a human, I would take this section (and some other sections below that are basically "PEP8") and just link to them so we aren't maintaining duplication of community standards. If the LLM can handle something like that, I think we should do that here. Given "agents", shouldn't they just be able to run ruff?

mdboom · 2025-12-08T14:50:48Z

cuda_core/style-guide.md

+
+#### Cython `cdef` Variables
+
+Consider prefixing `cdef` variables with `c_` to distinguish them from Python variables. This improves code readability by making it clear which variables are C-level types.


mdboom · 2025-12-08T14:51:30Z

cuda_core/style-guide.md

+
+```python
+LEGACY_DEFAULT_STREAM = C_LEGACY_DEFAULT_STREAM
+PER_THREAD_DEFAULT_STREAM = C_PER_THREAD_DEFAULT_STREAM


Additionally, related constants should be grouped into enums.

mdboom · 2025-12-08T15:10:01Z

cuda_core/style-guide.md

+    def __dealloc__(self):
+        """Automatic cleanup when object is garbage collected."""
+        DMR_close(self)
+```


Should we recommend that all such objects that require cleanup define a context manager, i.e. __enter__ and __exit__ methods?

Not for now. I can write a section explaining the current rationale in a separate PR if it is OK.

mdboom · 2025-12-08T15:11:54Z

cuda_core/style-guide.md

+
+5. **Use stream-ordered deallocation**: When deallocating buffers, use the appropriate stream for asynchronous cleanup to avoid blocking operations.
+
+6. **Track resource ownership**: Clearly document which objects own CUDA handles and are responsible for cleanup.


Can't we do better than just documenting, by specifying all objects contain a strong reference to their owner to prevent use-after-free errors etc.?

EDIT: Though maybe for a style guide this is too complex of an issue. For style, documenting that such a situation exists is the first requirement for actually patching these sort of holes.

FWIW, some of the things I'd like to document would more properly belong in a "Developer's Guide." I am therefore considering whether to expand the scope of this document. I'd restructure it into sections where "Style" is one section.

mdboom · 2025-12-08T15:18:23Z

cuda_core/style-guide.md

+
+### Inline Functions
+
+Use `inline` for small, frequently-called functions:


This is for Cython, which is not a modern C compiler ;) IME, sometimes the C compiler can see through it and make the right choice and other times not. The salient point for me is that we would ideally backup the choice of inline with a benchmark.

leofang · 2025-12-09T01:32:10Z

Merging my offline comment to here. My 2c:

If possible I’d like to make part (or all) of this PR rendered as Sphinx docs, for both internal and external contributors to use as developer guide
I’d also like all LLM to be able to read it when working on the codebase, including but not limited to: Cursor, CodeRabbit, Copilot, …

Andy-Jost added the documentation Improvements or additions to documentation label Dec 4, 2025

Andy-Jost self-assigned this Dec 4, 2025

Andy-Jost added the cuda.core Everything related to the cuda.core module label Dec 4, 2025

Andy-Jost requested review from cpcloud, keenan-simpson, leofang, mdboom, rparolin and rwgk December 4, 2025 20:50

Andy-Jost added this to the cuda-python 13-next, 12-next milestone Dec 4, 2025

Andy-Jost requested review from keenan-simpson and removed request for keenan-simpson December 4, 2025 20:50

Andy-Jost removed the request for review from keenan-simpson December 4, 2025 20:56

Andy-Jost removed this from the cuda-python 13-next, 12-next milestone Dec 4, 2025

Andy-Jost force-pushed the style-guide branch from 6d4003f to 193330f Compare December 4, 2025 21:34

Andy-Jost force-pushed the style-guide branch from 193330f to 8f6f10b Compare December 4, 2025 23:20

Andy-Jost added 2 commits December 4, 2025 16:34

Move style guide to cuda_core root directory

61994b2

Add Development Lifecycle section to style guide

9769d11

Document the two-phase development approach: - Phase 1: Start with Python driver implementation and tests - Phase 2: Optimize by switching to cydriver with nogil blocks Includes step-by-step conversion guide and before/after examples.

cpcloud reviewed Dec 5, 2025

View reviewed changes

leofang linked an issue Dec 6, 2025 that may be closed by this pull request

Update cuda.core developer guide #1101

Open

mdboom requested changes Dec 8, 2025

View reviewed changes


		Files in `cuda/core/experimental` must follow a consistent structure. The ordering of elements within a file is as follows:

		### 1. SPDX Copyright Header


		### 3. `__all__` Declaration

		If the module exports public API elements, include an `__all__` list after the imports and before any other code. This explicitly defines the public API of the module.


		2. Keep `.pxd` files minimal: Only include declarations needed for Cython compilation. Omit implementation details, docstrings, and Python-only code.

		3. Use `__all__` in submodules: Each submodule should define `__all__` to explicitly declare its public API.


		### Implementation Comments

		Add comments to explain complex logic or non-obvious behavior:


		### Prefer `cdef` for Internal Functions

		Use `cdef` functions for internal operations that don't need to be callable from Python:


		### Guidelines

		1. Placement: The copyright header must be the first lines of the file, before any imports or other code.


		6. Separate concerns: Use `.py` files for pure Python utilities, `.pyx` files for Cython implementations that need C-level performance.

		## Import Statements


		#### Cython `cdef` Variables

		Consider prefixing `cdef` variables with `c_` to distinguish them from Python variables. This improves code readability by making it clear which variables are C-level types.


		5. Use stream-ordered deallocation: When deallocating buffers, use the appropriate stream for asynchronous cleanup to avoid blocking operations.

		6. Track resource ownership: Clearly document which objects own CUDA handles and are responsible for cleanup.


		### Inline Functions

		Use `inline` for small, frequently-called functions:

Add comprehensive style guide for cuda/core/experimental #1316

Are you sure you want to change the base?

Add comprehensive style guide for cuda/core/experimental #1316

Uh oh!

Conversation

Andy-Jost commented Dec 4, 2025

Summary

What's Included

Key Principles

Benefits

File Added

Next Steps

Uh oh!

copy-pr-bot bot commented Dec 4, 2025

Uh oh!

Andy-Jost commented Dec 4, 2025

Uh oh!

Andy-Jost commented Dec 4, 2025

Uh oh!

rwgk commented Dec 5, 2025

Uh oh!

Andy-Jost commented Dec 5, 2025

Uh oh!

rparolin commented Dec 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cpcloud left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mdboom left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rparolin commented Dec 5, 2025 •

edited

Loading