-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add end_reason
auxillary field
#13
Comments
Would you be able to use buttery-eel with dorado-server? https://github.com/Psy-Fer/buttery-eel But yes, I can add a signal_positive to squigulator. This end_reason field in FAST5 used to change too many times in FAST5 and wasn't always reflective of the actual end reason, so that is why I did not include it in Squigulator. But given that now in POD5 end_reason has seemingly become a mandatory field, I will add it soon |
Thanks so much Hasindu. It's not a rush, mostly for convenience to be honest! I suppose it was a good motivator to familiarize myself with the I did look at |
I have very limited rust knowledge and slow5-rs is kindly developed and maintained by @bsaintjo who seems to have done a phenomenal job. Kudos to @bsaintjo! Yeh, simplify and usability is a key reason other than performance and file size, that I put effort into BLOW5 as described here. It is encouraging to hear that the effort is being paid off. I myself saved a lot of time by not having to spend time to fix each and every signal analysis tool that I developed, wherever a breaking change happens in POD5. @Psy-Fer is developing buttery-eel and we are open to hearing about complexities you encountered, and see if we can simplify it if possible (except complexities in the basecall-client from ONT that we do not have control over). One method is to provide buttery-eel with the latest dorado-server as a binary package which I call a snakeball, something I am keen to explore if there is a potential usecase, for instance this. |
Absolutely, kudos to @bsaintjo - really nice design patterns, learning a lot about how to handle C library integration! Really interesting idea to snakeball the dorado-server binary with On that note, I have long been wondering if there is a specification for the Guppy or Dorado server API that the underlying C (?) and Python client libraries use. I would love to cut these out from another code base that otherwise would not require the client package dependencies and could be compiled without much overhead. |
Hey, Yea documentation for the API you have to pull from the API itself. As for getting the builds, they are distributed from minknow builds, and then get shipped as a python lib on pypi that hook into the Dorado server API calls through a ipc/tcp client connection. It's not all that fun to develop, and frankly buttery-eel shouldn't exist. It's why has such a silly name. Ont should have just adopted slow5. But oh well. Buttery-eel can do whatever minknow live basecalling can do. So while standalone Dorado is nice, it's not really "production ready" till it ports into minknow. This has kept buttery-eel pretty reliable, especially now they have decoupled Dorado development and Dorado server releases. |
Hey James, long time, nice to get back in touch after me disappearing from social media completely :) I do miss your SciFi book recommendations ^^ I thought as much, will have to dig into the libraries at some stage - gave up the last time it came up in the interest of time, totally understand the "not fun to develop" part. Hard agree on the Slow5 adoption. Really cool work with wrapping it all up with |
Hello, I am also trying to convert the BLOW5 file to a POD5 using blue-crab, but I think I am getting the same error:
@esteinig I was wondering how did you add the header and field to the BLOW5 file to solve this issue? |
Hey @emics57 I can put a quick binary with converter together for you. Are you running this on a Linux system? |
Much appreciated!! Yes, on a Linux system. |
Also an update from @Psy-Fer, he has implemented blue-crab to set the end_reason to uknown if it doesn't exist and proceed with the conversion. See Psy-Fer/blue-crab#9. This way it not only will support squigulator-generated BLOW5 conversions, but also any BLOW5s converted from old FAST5 files when this end_reason attribute never existed. In the next release, I am going to implement the end_reason into squigulator too. |
Ah thanks @hasindu2008 @Psy-Fer better than running some sketchy binary 👀 I'll leave you with the update then! |
Yea be sure to try the dev branch to get that functionality while still being tested. I'll do a release and update pypi once the extensive testing has been done. |
OK, just cross reference as this discussion is scattered to blue-crab too |
Fantastic, very much appreciated @hasindu2008 and @Psy-Fer |
@hasindu2008, Thank you for the quick update!! The conversion is working for me now when using squigulator from the dev branch! |
Thanks so much for making such a useful tool @hasindu2008 et al. - it's been a delight to work with!
I unfortunately needed to do a Blow5 to Pod5 (...) conversion with
blue-crab
( ❤️ @Psy-Fer ) to use outputs in the latestDorado
version (0.5.3
). However, the conversion broke because of the missingend_reason
field. I simply added header and field to the Slow5, which enables the conversion.This is really just for convenience, but would it be possible to add the
end_reason
header and field e.g. with asignal_positive
enum variant to the output? I am unfortunately not familiar enough with C to add the field myself, but more than happy to open a PR if you can point me in the right direction :)The text was updated successfully, but these errors were encountered: