MinGW version of pysam #1274
Open
MinGW version of pysam #1274
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Pysam seems to be working at least under MSYS2 shell.
|
Benchmark had some errors. |
Couple of fixes to get pysam compiling on Linux
Couple more fixes for Linux
|
Please add a Windows build to the job matrix in .github/workflows/ci.yaml so that we can see this build in operation. |
Looks like I have no idea how to get the build job working nor do I have the time to learn to do so at the moment. |
|
Currently hacking away at this myself-the issue I'm running into running it under msvc python is that the cython extensions use posix-isms which...is not something you're really allowed to do. Seeing if I can't work around it. edit: its kinda worse than that-the shared model doesn't quite work because you can't link a .a output from gcc using cython/msvc. |
I think pysam needs major patching to work with MSVC and/or clang-cl. There is a MSVC port of htslib and that might be a good starting point. https://github.com/refresh-bio-sandbox/htslib-msvc/tree/develop |
|
the other way to go about doing it would be to use something like CMake's GNUToMS and to link it as such-probably easier, honestly. edit: and as luck would have it htslib's makefile already spits out a .lib if you use build-shared |
|
Okay, so I was mostly getting around portability issues with simple find-replace and a few mingw headers (mostly strcasecmp/_stricmp and STDIN_FILENO/ _fileno(stdin) alongside some symbol changes (specifically the data variable exports from htslib get mangled) and this worked until I ran into samtools coverage.c. Apparently msvc doesn't support VLAs. Because why would it? Gonna be a more involved port than I'd like. |
|
If you can, please share your progress on Github or elsewhere. |
|
Oh i will, don't worry.
…________________________________
From: teepean ***@***.***>
Sent: Tuesday, March 18, 2025 2:07 PM
To: pysam-developers/pysam ***@***.***>
Cc: Andrew Robbins ***@***.***>; Comment ***@***.***>
Subject: Re: [pysam-developers/pysam] MinGW version of pysam (PR #1274)
If you can, please share your progress on Github or elsewhere.
—
Reply to this email directly, view it on GitHub<#1274 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACFNSCLC3UA3XNYNY737BXL2VBOFPAVCNFSM6AAAAABRH2AKW2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMZUGI3TINBQGI>.
You are receiving this because you commented.Message ID: ***@***.***>
[teepean]teepean left a comment (pysam-developers/pysam#1274)<#1274 (comment)>
If you can, please share your progress on Github or elsewhere.
—
Reply to this email directly, view it on GitHub<#1274 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACFNSCLC3UA3XNYNY737BXL2VBOFPAVCNFSM6AAAAABRH2AKW2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMZUGI3TINBQGI>.
You are receiving this because you commented.Message ID: ***@***.***>
|
AFAIK clang-cl supports VLAs. |
|
Most likely I will only be interested in merging an MSYS2-based approach such as @teepean's. What you're describing for MSVC support is too invasive for a niche platform. |
|
the primary issue there is that msys2-based python is...not widely used. the goal if i can get this to work is for it to be minimally invasive, don't worry! |
AIUI MSYS2 produces native executables without unusual dependencies. If this is the case, or if pysam uses delvewheel when building wheels, this is irrelevant. |
|
I might be operating on outdated information here but my understanding was that python extensions proper need to link the pythonXY.lib file for MSVC python, which isn't impossible, just a pain. Will keep posted. edit: actually this may be easier than i thought |
|
okay, the good news is that i have it building against MSVC python w/ no major modifications. the bad news is... `ModuleNotFoundError: No module named 'pysam.libchtslib' |
With MinGW builds this means that the module cannot find all of it's dependencies. |
|
yeah, I checked that. Delvewheel and all. No dice
|
|
https://github.com/theAeon/pysam/tree/pysam-mingw if you want to give it a try |
Any instructions on how to build this one? |
|
|
This produces a build with gcc not MSVC or am I missing something here? |
|
linked w/ msvc python |
|
In that case the resulting pyds depend on ucrt64 dlls. |
|
which i put in the wheel with delvewheel (specifically give it a go, it won't work |
|
I built pysam accidentally with clang64 instead of ucrt64 but decided to go with that. Here's the delvewheel version. Basic functionality seems to be working. https://drive.google.com/file/d/1c9vpNIpwWtFVkLdQxFoXs-wcaIKPoodg/view?usp=sharing |
|
Here's a test log. https://gist.github.com/teepean/d50974cffa9f0a4f38da99958eaa70ab |
Huh. Wonder why it wasn't working on end. |
You mean why it wasn't working on your end? Did you test the whl I linked earlier? |
On my end-and I was referring to my delvewheel run. I'll give that a try shortly. |
bizarre |
|
I am using official Python 3.13.1. I am going to upgrade and see what happens. |
|
Works just fine. You could use Procmon to see which one of the DLLs is not found. |
|
looks like its trying to load copies of libbz/libcrypto/libcurl under an un-mangled name? |
|
It loaded, but only after my pysam.libs folder was made to look like this. Probably safe to write off as something wrong with my install. Probably. |
|
oh, its way stupider than that. there was an extra hts-3.dll in pysam/ that wasn't getting deleted by pip uninstall and was messing everything up. disregard. let me fix ci while i'm at it |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Build tested on MSYS2 ucrt64 environment.
Test results: