Skip to content

Commit

Permalink
Beautifications on the ROADMAP
Browse files Browse the repository at this point in the history
  • Loading branch information
FrancescAlted committed Aug 14, 2019
1 parent 78b5cd7 commit 779450d
Show file tree
Hide file tree
Showing 2 changed files with 18 additions and 5 deletions.
14 changes: 14 additions & 0 deletions RELEASING.rst
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,20 @@ where instructions on how to intensively test (and benchmark) Blosc
are given.


Update documentation
--------------------

Go the the `blosc-doc <https://github.com/Blosc/blosc-doc>`_ git repo and update the `README.rst`
and `blosc2.h` and do a commit::

$ cp ../c-blosc2/README.rst src/c-blosc2_files/README.rst
$ cp ../c-blosc2/blosc/blosc2.h src/c-blosc2_files
$ git commit -a -m"Updated documentation for release X.Y.Z"
$ pit push

After pushing, a new documentation will be rendered at: https://blosc-doc.readthedocs.io/en/latest/
Check that it contains the updated docs.

Tagging
-------

Expand Down
9 changes: 4 additions & 5 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,15 +15,15 @@ Throughout this document we are going to use the next conventions:
Existing features
-----------------

Right now (August 2019), the next features are already implemented (although they may require some refactoring or most tests):
Right now (August 2019), the next features are already implemented (although they may require some refactoring or more tests):

* 64-bit containers. The first-class container in C-Blosc2 is the `super-chunk` or, for brevity, `schunk`, that is made by smaller containers which are essentially C-Blosc1 32-bit containers. The super-chunk can be backed or not by another container which is called a `frame` (see later).

* More filters: besides `shuffle` and `bitshuffle` already present in C-Blosc1, C-Blosc2 already implements:

- `delta`: the stored blocks inside a chunk are diff'ed with respect to first block in the chunk. The idea is that, in some situations, the diff will have more zeros than the original data, leading to better compression.

- `trun_prec`: it zeroes the least significant bits of the matissa of float32 and float64 types. When combined with the `shuffle` or `bitshuffle` filter, this leads to more contiguous zeros, which are compressed better.
- `trun_prec`: it zeroes the least significant bits of the mantissa of float32 and float64 types. When combined with the `shuffle` or `bitshuffle` filter, this leads to more contiguous zeros, which are compressed better.

* A filter pipeline: the different filters can be pipelined so that the output of one can the input for the other. A possible example is a `delta` followed by `shuffle`, or as described above, `trunc_prec` followed by `bitshuffle`.

Expand All @@ -49,16 +49,15 @@ Actions to be done

* Documentation: utterly important for attracting new users and making the life easier for existing ones. Important points to have in mind here:

- Quality of API docstrings: is the mission of the functions or data structures clearly and succintly explained? Are all the parameters explained? Is the return value explained? What are the possible errors that can be returned?
- Quality of API docstrings: is the mission of the functions or data structures clearly and succinctly explained? Are all the parameters explained? Is the return value explained? What are the possible errors that can be returned?

- Tutorials/book: besides the API docstrings, more documentation materials should be provided, like tutorials or a book about Blosc (or at least, the beginnings of it). Due to its adoption in GitHub and Jupyter notebooks, one of the most extended and useful markup systems is MarkDown, so this should also be the first candidate to use here.

* Wrappers for other languages: Python and Java are the most obvious candidates, but others like R or Julia would be nice to have. Still not sure if these should be produced and maintained by the Blosc development team, or leave them for third-party players that would be interested.

* It would be nice to use [LGTM](https://lgtm.com), a CI-friendly analyzer for security.

* Adding support for [buildkite](https://buildkite.com) as another CI would be handy.

* Add support for `buildkite <https://buildkite.com>`_ as another CI would be handy because it allows to use on-premise machines, potentially speeding-up the time to do the builds, but also to setup pipelines with more complex dependencies and analyzers.

Outreaching
-----------
Expand Down

0 comments on commit 779450d

Please sign in to comment.