SNOW-1799654: Improve MANIFEST.in File for Better Packaging Consistency

[shashankrushiya]

What is the current behavior?

Currently, the MANIFEST.in file for the snowflake-snowpark-python package does not fully follow best practices for Python packaging. Essential files like README.md, LICENSE.txt, and source files with typing information are already included, but there are a few inconsistencies in the structure, and some unnecessary files are excluded in a less organized way.

What is the desired behavior?

The desired behavior is to have a streamlined and well-structured MANIFEST.in file that includes all essential documentation and source files while excluding files only needed for development (e.g., .pre-commit-config.yaml, CONTRIBUTING.md, license_header.txt, etc.). This includes adding support for *.pyi files within src/snowflake/snowpark, which helps ensure compatibility and type safety for developers using type hints.

How would this improve snowflake-snowpark-python?

This update would improve the package by:

1. Ensuring that essential files and source code are consistently included, allowing for a more predictable and complete package for
   users.
2. Simplifying the distribution by excluding unnecessary files, which reduces the package size and avoids potential confusion with
   non-essential files.
3. Enhancing the development experience with better-organized files, leading to a cleaner, more manageable project structure.

References, Other Background

This request is motivated by standard best practices in Python packaging and aims to improve both the user and developer experience by refining the file inclusion and exclusion rules in the MANIFEST.in.

These changes are intended to align the snowflake-snowpark-python package with industry standards for efficient, minimal, and complete packaging.

[sfc-gh-sghosh]

Hello @shashankrushiya ,

Thanks for raising the concern, so are you saying to include *.pyi files ( python interface files) for static typing to ensure type safety?

Regards,
Sujan

[shashankrushiya]

Hi Team, Thanks for getting back to me! Yes, that's exactly what I meant. Including *.pyi files in the package would be really helpful for developers who use static typing. It ensures better type safety and makes tools like mypy more effective when working with snowflake-snowpark-python. This small change can make a big difference in improving the developer experience. Best regards, Shashank Rushiya

…

On Mon, Nov 18, 2024 at 9:41 AM Sujan Ghosh ***@***.***> wrote: Hello @shashankrushiya <https://github.com/shashankrushiya> , Thanks for raising the concern, so are you saying to include *.pyi files ( python interface files) for static typing to ensure type safety? Regards, Sujan — Reply to this email directly, view it on GitHub <#2608 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A56PJDGROWA4WAEY4RQC2U32BFSFPAVCNFSM6AAAAABRUEJYJWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIOBRHEYDQNRYGM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[shashankrushiya]

Hi @sfc-gh-sghosh ,

I noticed the "status: information needed" label and wanted to ensure I address any questions or missing details.

To clarify, this PR focuses on refining the MANIFEST.in file to improve the packaging process for snowflake-snowpark-python. Specifically, it ensures:

1. Inclusion of all essential files (e.g., README.md, LICENSE.txt, NOTICE) for a complete package.
2. Support for *.pyi files to provide better static typing and compatibility with tools like mypy.
3. Exclusion of unnecessary development files (e.g., .pre-commit-config.yaml, CONTRIBUTING.md) to keep the package clean and minimal.

If there’s anything specific that requires further information or clarification, please let me know, and I’ll address it promptly.

Looking forward to your feedback!

Best regards,
Shashank Rushiya

[sfc-gh-sghosh]

Thank you @shashankrushiya for the update and clarification, will update you further after discussion with team.

Regards,
Sujan