Minor bugfixes

Bugfixes

• Bug in slicing views using tuples #77
• Fix: when attempting to connect to non-existing file, an empty file would be created

Bugfixes, docs

Bugfixes and better docs

Fixes many minor and not so minor bugs.

Greatly improved documentation at loompy.org

Updated documentation

Minor release only affecting the docs.

More sparsity support, empty layers & experimental plotting features

Any layer can be created from sparse matrix

This now works:

G = 1000
C = 100
S = sparse.eye(G, C)
with loompy.connect("test.loom") as ds:
    ds["layer"] = S

Fixes #66.

Create empty file

loompy.new() creates an empty loom file, and returns it as a context manager. The file can then be populated with data. This is especially useful when you're building a dataset incrementally, e.g. by pooling subsets of other datasets:

with loompy.new("outfile.loom") as dsout:
    for sample in samples:
        with loompy.connect(sample) as dsin:
            logging.info(f"Appending {sample}.")
            dsout.add_columns(ds.layers, col_attrs=dsin.col_attrs, row_attrs=dsin.row_attrs)

As a consequence, create_append() is now deprecated.

Fixes #42.

Experimental plotting features

ds.pandas() to return a Pandas DataFrame for the whole matrix, or selected parts. The interface is intended to simplify plotting, since many plotting libraries take Pandas as input. The interface is experimental, and e.g. lacks support for layers.

ds.embedding() and ds.embeddings() to find attributes that are >1-dimensional. Again intended to support plotting, by making it easy to find the X/Y coordinates without knowing if they are stored as TSNE, PCA or something else. The interface is liable to change (in particular, I'd like to find a shorter name than "embedding").

Two useful colormaps: loompy.zviridis is a zero-inflated version of viridis, good for plotting zero-inflated data. loompy.cat_colors() is a function that generates N distinct colors, in pleasing and distinguishable hues, for large N.

Contributes to #62.

Compatibility with Seurat & create empty layers

Compatibility with Seurat & loomR

In some cases, Seurat would create loom files with attributes being variable-length ascii. This technically violates the loom specification, but what's worse is that loompy would read them as byte arrays. We now handle such strings gracefully and they are returned as arrays of string objects (supporting unicode).

Create empty layers

Previously, there was no way to create an empty layer without supplying a dense matrix. This would cause problems when you wanted to add a larger-than-RAM layer to an existing file. We now support an elegant syntax for creating an empty layer, directly on disk, by assigning a data type to the layer name. For example:

with loompy.connect("filename.loom") as ds:
    ds.layers["intronic"] = "int16"

Or, using the shorthand syntax directly on the connection object:

with loompy.connect("filename.loom") as ds:
    ds["intronic"] = "int16"

Once the layer has been created, you can assign values to (parts of) the layer, building it up incrementally.

Bugfixes

Note: If you're using sparse matrices to create loom files, this update fixes a nasty performance bug (see #48)

Bugfixes

• Exception when assigning a layer directly on the connection object:

with loompy.connect("filename.loom") as ds:
    ds["layername"] = m

• Severe performance bug when creating from large sparse matrix (issue #48)
• Error in example of valid file format (issue #44)
• Improved handling of connections opened in readonly mode (see commit d92bac9)

Bugfixes and minor new features

Bugfixes

• Bug in sparse() caused it to load only the first 1000 cells

New features

• scan() method now accepts boolean mask arrays to select items (rows/cols)
• Opening a file that uses old-style row_edges and col_edges automatically adds row_graphs and col_graphs

Bugfixes and __version__ attribute

Bugfixes and minor new features:

• loompy.__version___ attribute (gives version of loompy package)
• Better handling of global attributes when combining files
• Make it possible to delete global attributes

Experimental new file feature: file spec version stamp

LOOM_SPEC_VERSION HDF5 attribute on the root group. Use it like this:

with loompy.connect(filename) as lc:
    if "LOOM_SPEC_VERSION" in lc.attrs:
        print("file version: " + lc.attrs.LOOM_SPEC_VERSION)
    else:
        print("file version: less than 2.0.1")

Note: this is experimental, and is a proposed new feature of the loom file format specification. As such, the attribute is not guaranteed to exist, and you must always check for it before trying to read it. If it doesn't exist, then the loom file spec version is assumed to be less than "2.0.1" by default.

Note: the LoomSpecVersion is not the same thing as the loompy package version.

Bugfix release

Finally fixed sparse matrix support, so that this code is verified to work:

import numpy as np
import loompy
import scipy.sparse as sparse
filename = "test.loom"
matrix = sparse.coo_matrix((100, 100))
row_attrs = { "SomeRowAttr": np.arange(100) }
col_attrs = { "SomeColAttr": np.arange(100) }
loompy.create(filename, matrix, row_attrs, col_attrs)

(CSR, CSC and COO matrices are all supported and tested).

Also updated the docs with the example above.

Bugfix release

What's new

• Fixed a bug where create() would not recognise sparse matrices