vs. rust htslib

[Kentalot]

Hi,

I was looking at using rust for bioinformatics processing, but was wondering what the main difference between noodles and rust-htslib is from a practical user perspective? Is noodles supposed to be faster or simply pure rust or anything? htslib is used widely and well tested so seems like rust-htslib can do almost everything noodles can do, no? Is noodles significantly faster or anything? I am aware that rust-htslib doesn't have bed functionality, though they do have rust-bio that does do beds? Any comments would be appreciated, especially by the authors. Thanks!

[zaeleus]

The primary motivation of noodles is to attempt to provide compliant implementations of well-known large scale genomics specifications, particularly hts-specs. This is quite different from existing implementations, which tend to read and write particular dialects of the formats, either for historical reasons or ignorance of the specifications. As such, it hinders interoperability, reducing confidence when sharing data or collaborating among users of different tools.

When strictly dealing with I/O, yes, htslib and noodles are rather similar. Both can read and write records in the supported alignment (SAM, BAM, and CRAM) and variant (VCF and BCF) formats, and both allow random access using an index.

The decoders/parsers, however, have different ideologies. htslib is very lenient; whereas, noodles is far more strict. For example, take the following alignment record (207.bam):

@HD	VN:1.6
🍜	4	*	-7	255	4B	*	-7	0	ACGT	}~��	æ:i:0	æ:i:1

I'd argue that this is not a valid record. The read name is non-ASCII, the positions and quality scores are out of range, the CIGAR operation is invalid, and the auxiliary data tags are both invalid and duplicated. Yet, htslib/samtools will read (and rewrite) this without error.

$ samtools version | head -1
samtools 1.18
$ samtools view --no-PG --with-header 207.bam
@HD     VN:1.6
🍜      4       *       -7      255     4B      *       -7      0       ACGT    }~      æ:i:0   æ:i:1
$ echo $?
0

The typical record reader in noodles (i.e., bam::Reader::read_record) will fail reading this, stopping at the first invalid field.

$ cargo run --example bam_view 207.bam
Error: invalid position

Caused by:
    0: invalid input
    1: out of range integral type conversion attempted

But perhaps this fine; perhaps users want nonsensical and/or bad data. htslib made a choice to prefer performance over conformance. (noodles also provides an alternative lazy/raw record reader, which allows for per field decoding but does not allow rewriting them.) This is likely the major practical difference users will encounter in noodles when compared to htslib, i.e., specification stringency.

Some extra notes:

• noodles' public API is less terse. They're not identical in usage, but see add_header.c vs sam_reheader.rs, read_bam.c vs bam_view.rs, etc. rust-htslib does have the added benefit of providing a nice facade over htslib's C API though.
• Nearly all the crates in noodles are pure Rust implementations. noodles-cram relies on libbz2 and liblzma, and noodles-bgzf can be optionally be built with libdeflate.
• Positions in noodles are 1-based ([start, end]), and htslib is 0-based ([start, end)).
• noodles should be competitive with htslib in performance, but remember that their decoders/parsers are not performing the same work.

[Kentalot]

Great, thanks for the detailed reply. This makes sense and I think people do care a lot about valid data. I will evaluate our needs and decide which one works best for our project. Thanks so much!