Skip to content

CHANGELOG.md

Latest commit

 

History

History
756 lines (487 loc) · 58.4 KB

CHANGELOG.md

File metadata and controls

756 lines (487 loc) · 58.4 KB
Raw

Changelog

[Unreleased]

💥 Breaking changes

  • SortedKeySet and SortedKeyDict don't expose anymore their key attribute, as this is an implementation detail.
  • {SortedKeyList, SortedList}::{add, update} methods have been replaced by append and extend, respectively. Unlike the original sortedcontainers library, who raise runtime errors if those two are called. Allow to be more compliant with the MutableSequence protocol.
  • {SortedKeyList, SortedList}::insert does not raise runtime errors anymore, and simply forward to append. Index is ignored. Allow to be more compliant with the MutableSequence protocol.

No more identity function as default key for "key containers"

"keys" sorted containers (i.e SortedKeyList, SortedKeySet, etc...) now require an explicit key argument, instead of providing by default an identity function.

Moreover, key is now the first argument, followed by the iterable argument, and they are both positional-only.

This allows performance improvements, simplify internal implementations, but most importantly, aligns more with the API choice of pyochain, which, unlike the original sortedcontainers library, provides those "key containers" as explicit classes, instead of dynamically creating them depending on the signature.

Because, frankly, why would you create a SortedKeyList with an identity function as key, when you can just use a SortedList instead?

🚀 Performance improvements

  • Massive global improvements to SortedList and SortedKeyList, and by extension sorted sets/dicts/views who use them internally. Detailed benchmarks will be provided in the near future.

  • pprint was used internally in many __repr__ implementations, which is no longer the case. The "raw" repr of inner objects (e.g list, set, etc...) is now used, which means we swap a Python call with a CPython one.

args concatenation

When you domy_iter.for_each_star(f, *args), each element T of my_iter need to be passed first to f as the first argument, followed by the unpacked args.

This entails reconstructing a new data structure to hold said arguments.

The related logic has been optimized and fully refactored, with significant performance improvments, especially when the number of args is small (1 to 8), which is the most common use case.

See below for the benchmark results, with the number of args in the left column, the number of items in the Iterator in the header row, and the relative performance improvement in the cells.

2.00x means that the new implementation is twice as fast as the previous one.

- 10 100 1_000 10_000 100_000
2 1.25x 1.37x 1.40x 1.43x 1.43x
4 1.28x 1.37x 1.37x 1.37x 1.39x
8 1.31x 1.45x 1.47x 1.47x 1.47x
16 1.21x 1.26x 1.26x 1.25x 1.22x
32 1.00x 1.17x 1.19x 1.20x 1.19x
  • ``PyoIterator::map_windows_star` is 1.05x to 1.26x faster across all sizes tested, (small and big iterators, small and big window sizes). A single regression has been noted at 100k items with a window size of 32, which is planned to be fixed in a future release.

✨ Enhancements

  • typing: Relaxed the collector input type of PyoIterator::collect. The constraint on the return type R: Collection[Any] was artificial, and was preventing to use collect on functions or types who indeed consume the Iterator, but weren't strictly speaking a Collection (e.g polars DataFrames). In python, the FromIterator equivalent is simply Callable[[Iterator[T]], Any], and this is now reflected in the typing of collect.
  • typing: Option::{is_some, is_none, unwrap, unwrap_or, unwrap_or_none} now have overloads to statically return the narrowed type when the variant is already known.
  • typing: Result::and_then handles a change of Error type in the closure, and returns the correct Result type. i.e Result[T, E].and_then(f: Callable[[T], Result[U, F]]) -> Result[U, E | F].

✨ Enhancements

  • typing: Ensured that all sorted containers classes are marked with typing::{type_check_only, final} when appropriate.

🐞 Bug fixes

  • typing: PyoIterator::{map_windows, map_windows_star} now correctly handle functions with varargs.
  • typing: PyoIterator::for_each_star now correctly handle functions with varargs and ParamSpec.
  • typing: Iter::__new__ missed overloads, which caused false positives with type checkers, for example, when providing a single Iterable argument. This is now fixed.

🔄 Refactors

  • Many changes to the internal rust structure to improve readability and reduce code-gen (and by extension, compile time).

📦 Build system

  • Added panic="abort" flag on release builds to reduce binary size and improve performance.
  • Improved caching of CI workflows for performance.

🛠️ Other improvements

  • Website: The return sections of the functions/methods docstrings is now correctly rendered thanks to @tecnolgd contribution in #95

  • Profiler scripts: Added a dedicated typer app to handle, analyze and plot benchmark results. Currently focused on sortedcontainers comparison, but can be extended to more in the future.

  • Dev check scripts: Centralised the linting/formatting/etc... commands in a single entry point.

[0.28.0] - 2026-08-31

🏆 Highlights

  • Thanks to @tecnolgd contribution in #92, Range is now aligned with its python counterpart constructor: Range(10) creates a range from 0 to 9, while Range(1, 10) creates a range from 1 to 9. The step argument is optional and defaults to 1.

Flexible constructors

core::{Dict, Iter,Range, Vec, Seq, Set, SetMut}, and collections::{StableSet, Deque} now have flexible constructors: instantiate with no arguments, a variable number of positional elements, or a single iterable.

For example, Vec(1, 2, 3) or Vec([1, 2, 3]) will create the equivalent of [1, 2 ,3], while Vec() or Vec([]) will create an empty Vec.

Specialized constructors are available: from_iter for a single Iterable, of for a variadic number of args (or kwargs for Dict), or wrap, which replaces from_ref for concerned classes (e.g Vec.wrap(list())).

Use them if you want to keep performance identical to the previous version, or if you prefer explicitness.

Note that the overhead of the new constructors is very small, and the performance difference is negligible for most use cases. For example, the overhead of the attribute retrieval of Vec.from_iter is almost equal to the overhead of the Vec constructor itself. Only in situations like .iter().map(Seq.from_iter) can the overhead be noticeable, if you create a lot (+1k) of small (~10 elements) Seq instances. As always, benchmark your code before making any changes.

💥 Breaking changes

  • core::Peekable can only be constructed from PyoIterator::peekable.
  • {SortedList, SortedKeyList}::load is now fully private

🚀 Performance improvements

  • PyoIterator::map_juxt is now 1.5× faster when provided with 1 function, 1.29× faster with 4–16 functions, and 1.13× faster with 64 functions. Results are consistent across iterables of sizes ranging from 10 to 10,000 items.
  • Pyoiterator::zip_longest is now between 1.6x to 1.8x faster when provided with a single iterable, across tested sizes(10, 100, 1_000 and 10_000). Expect faster speed-ups with more arguments, as it's the overhead of creating the Option values that has been greatly reduced.

🆕 New features

  • Result::{__eq__, __ne__} are now implemented, allowing to compare Result instances for equality.

🐞 Bug fixes

  • Err::flatten was actually not implemented. To prevent similar issues, the code for monads has started to be refactored to use py_abc macro.

✨ Enhancements

  • Added Never return types to a few methods of PyoIterator to clearly mark situations that will raise an error at runtime, and to help static type checkers to detect unreachable code.

🛠️ Other improvements

  • Relaxed the Callable return type for Option::filter to object, since all python objects can be evaluated for truthiness.
  • Option::__bool__ return Never to comply with python typing standards regarding "always-failure" functions.
  • Ok and Err now have their generics types to Any as default. This allow to do Ok[int, str] without seeing it inferred as Result[int, Any], it now becomes Result[int, str] as expected.
  • Various clean-ups and refactors in the Rust codebase, to improve readability and maintainability.

[0.27.0] - 2026-08-19

NOTE: the performance improvements and benchmarks are very likely understated, as they were done when the Rust migration was still partial.

🏆 Highlights

  • Full migration of the whole codebase to Rust
  • abc::PyoIterator::peekable has been completely refactored and now aligns with Rust std Iterator, with the methods peek, next_if, next_if_eq, and next_if_map, all implemented in Rust.
  • documentation: All docstrings examples now use markdown code blocks instead of doctests, significantly improving readability. Many thanks to our contributor @tecnolgd for his help on this matter.
  • typing: The variance of generics for all classes is now correctly handled and align with python stdlib. For example, PyoMapping, Seq or Set are now covariant, whilst Dict or Vec stay invariant.
  • collections::PyoCounter, pyochain version of python stdlib collections::Counter. Herit from PyoMutableMapping instead of dict, but behaves the same way.
  • collections::{HeapMax, HeapMin}, pyochain reimplementation of python stdlib heapq module with an OOP, fluent API.
  • Full port of the sortedcontainers library to pyochain, rewritten with native rust data structures, and fully typed. Credits to Grant Jenks for the original implementation, as well as the maintainers of the corresponding stubs package.

💥 Breaking changes

PyoIterable is now only here for the iter method, and doesn't provide methods that implicitely convert the Iterable into a PyoIterator anymore. last and first, depending on the concrete subclass, were either redundant (first for PyoIterator who already has next) or not really useful (last for non-ordered Collections).

  • API change: PyoIterable::first has been deleted. This impacts all non-sequence types (PyoSequence is unchanged). If you need the old behavior on an Iterator or a Collection, call .iter().next().unwrap() instead.
  • API change: PyoIterable::last has been moved to PyoIterator and PyoSequence as separate methods without class inheritance. This impacts all custom PyoIterable subclasses who aren't Iterators nor Sequences. If you need the old behavior (e.g calling last on a set-like collection), call iter().last() instead.
  • API change: PyoIterator::batch has been renamed to PyoIterator::batched.
  • Method removal: PyoIterator::array_chunks has been removed. Use PyoIterator::batched instead.
  • API change: PyoIterator::map_with function argument now need to be the first argument, followed by the various iterables. It was previously a kword only argument at the end. Swap the order at call sites to migrate your code.
  • Removal: PyoIterator::insert has been removed. Replace y.insert(x) by Iter.once(x).chain(y) to migrate your code. This makes thing clearer at both reading-order level, and semantics level, as this avoid treating an Iterator as "sort of" mutable collection.
  • Removal: Original PyoIterator::repeat has been removed, and PyoIterator::from_repeat renamed to repeat, i.e from_repeat replaces the old repeat. The original had complex semantics, niche use cases, without a real performance/memory benefit. To get the same behavior, use something like my_iter.collect(Seq).pipe(lambda it: Iter.repeat(it, n)).map(lambda x: x.iter()). This also better align with rust Iterator::repeat semantics and naming.
  • Removal: Iter::__bool__ has been removed. Use PyoIterator::peekable::__bool__ instead. This avoid implicit tee use.
  • Removal: Iter::{from_ref, cloned} have been removed. Use a, b = x.tee() instead (for cloned), or a, b = Iter(x).tee() (for from_ref), where a is the original Iterator, and b the cloned one.
  • Dunders methods (like __add__, __mul__, __getitem__, etc...) now return new instances of the same class, instead of the underlying data structure when applicable, i.e Vec + Vec returns a Vec instead of list.
  • imports: views-like classes are now back to being only imported from abc module, to stay consistent with python stdlib.
  • Removal: inner attribute is now a private implementation detail of pyochain, rather than something accessible from the public API. If you are confronted by code that refuses to handle, say, a MutableSequence instead of a list, 99% of the time, it's a good time to change it (or raise an issue to the author of the code). Python is meant to use duck typing, and def foo(x: list): is, again, not a valid pattern, at runtime or for static typing, 99% of the time. In any case, you can simply do x.pipe(list) if needed, for example.

🆕 New features

  • {Dict, SetMut, StableSet, Vec}::copy to create shallow copies of the instance, just like x.copy() for their respective underlying builtins.
  • PyoSized, PyoReversible, and PyoContainer now inerhit from Checkable, instead of relying on mixins heritance.
  • get_item is now a PyoMapping method instead of a PyoMutableMapping one.
  • Dict, Set and SetMut are now fully compliant with their underlying data structure regarding the methods available.
  • SetMut::difference_update.

✨ Enhancements

  • typing: added overloads to PyoIterator::batched to return precise tuple types for batch sizes up to 5 if strict is True.
  • typing: added overloads to PyoIterator::product to return precise tuple types for up to 10 iterables.
  • API: PyoIterator::accumulate is now aligned with itertools, with the possibility to not provide a function, in which case the default is to use addition.
  • API: PyoIterator::product missed it's repeat argument. This is now fixed.
  • typing: Iter, Option, Result, Peekable, Seq, Set, and abc::{PyoIterable, PyoIterator, PyoCollection, PyoSequence, PyoSet} types are now covariant in their generic type.
  • typing: PyoSequence::get now has correct generic types for slice overload.
  • typing: fixed overloads of __get_item__ for Vec, Seq and Range.
  • typing: added overloads to PyoIterator::filter_star to return precise tuple types for up to 3 elements when provided with a closure returning a TypeIs or TypeGuard specifying the narrowed tuple.
  • typing: relaxed closure return type for PyoIterator::{filter, filter_star, filter_false} to object, since all python objects can be evaluated for truthiness.
  • typing: The variance of generics for immutable classes (abstract and concrete) is now correctly handled and align with python stdlib. For example, PyoMapping, Seq or Set are now covariant, whilst Dict or Vec stay invariant.
  • typing: Dict::from_object now returns Dict[str, object] instead of Dict[str, Any] to avoid silencing errors with strict type checkers (e.g Pyrefly, Basedpyright) who forbid the use of Any in generic types. You can always use typing::cast if you need specific attribute access without runtime check AND no type checker errors.
  • typing: Result and Option method transpose give less false positives now.
  • typing: Thanks to covariance, PyoIterator::flatten should not give false positives anymore, since it seems there's no need anymore to manually maintain various overloads for nested PyoIterator types.
  • typing: PyoIterator::chain now has overloads to return precise tuple types for up to 5 iterables, and handle covariance in all cases.
  • No-copy behavior for Seq when created from an already existing Seq, matching Python tuple behavior. No more need to use Seq(seq.inner) to keep the same object, Seq(seq) is enough. Note that this doesn't impact performance much, since the underlying tuple, where the data is stored, wasn't copied anyway, and the Seq wrapper is very lightweight in itself. This however help to make Seq behave more like tuple, and avoid confusion when using is operator on two Seq instances created from the same underlying data.

🐞 Bug fixes

  • Aligned abc::PyoMappingView heritance and generics logic with python builtins.
  • PyoIterator::all_equal without key argument didn't have a default None value altough present in stubs, causing unexpected TypeError.
  • Deleted PyoIterator::_from_iterableclass constructor, which caused issues for subclassing. As of now, Each method return an Iter upcasted to PyoIterator. Expect NO change in behavior. In a future release, each Iterator will have a dedicated struct (as long as it's faster than python builtins), hopefully allowing further performance improvements.

🚀 Performance improvements

Underlying structure delegations

Instead of relying on automatic ABCs implementations, concrete classes now delegate to their underlying structure systematically.

  • abc::Reversible::rev got huge performance gains on concrete classes (Range, Vec, etc...), up to 28x faster, by implementing __reversed__ dunder, which now return the C level Iterator directly, instead of relying on the abc auto implementation.
  • Vec::__iter__ has received the same type of optimization as rev. A quick benchmark with timeit show improvements between 25x to 45x faster on sizes ranging between 10k to 100k items. Dict::__contains__ has received the same changes, but the performance isn't really changed, as the default ABC implementation is not that different from an optimized one (at least with a dict).
  • append, clear and extend received the same optimization on Vec and Deque.

PyoIterator methods

Many methods have been migrated to Rust.

See the table below for the performance improvements, with the number of items processed in each Iterator when testing.

Note that for unpack_into, even if it's worse on a relative basis on small Iterator's, the median absolute speed improvement on larger ones more than makes up for it, since on 10 items, it's 2.6 us (new) vs 2.42 us (old), while on 10k items, it's 229 us (new) vs 253 us (old).

What was observed is that the upper values outliers are much less frequent, especially on smaller sizes, where they have been reduced by a factor of 4.

The performance improvements on small Iterators are expected, but the slowdown (albeit minor) on large Iterators is very surprising.

There's less outliers, but their magnitude is worse, which can't be explained easily.

Name 10 items 100 items 1_000 items 10_000 items Note
arg_max 1.26x 1.78x 3.40x 4.15x Low items counts are likely higher after full Rust migration
arg_min 1.25x 1.73x 3.37x 4.18x Low items counts are likely higher after full Rust migration
arg_max_by 1.24x 1.58x 2.70x 3.10x Low items counts are likely higher after full Rust migration
arg_min_by 1.29x 1.62x 2.59x 3.03x Low items counts are likely higher after full Rust migration
unpack_into 0.93x 0.99x 1.09x 1.11x Low items counts are likely higher after full Rust migration
zip_longest 2.83x 4.57x 4.64x 4.46x Due to option creation in Rust
unzip 1.61 2.80x 3.82x 4.09x Due to tuple access in Rust
all_equal 0.99x 1.02x 0.98x 1.01x Identical perf, Low items counts are likely higher after full Rust migration
try_collect 1.26x 1.08x 1.01x 1.00x Due to Vec creation in Rust, Low items counts are likely higher after full Rust migration
partition 1.24x 1.05x 1.02x 1.02x Due to Vec creation in Rust, Low items counts are likely higher after full Rust migration
is_sorted 1.12x 1.07x 1.01x 1.00x Due to default param now in Rust.
group_by 1.70x 2.30x 2.39x 2.37x -
map 1.08x 1.03x 1.00x 1.04x -
accumulate 1.11x 1.05x 1.04x 1.00x -
reduce 1.05x 1.01x 1.00x 1.00x -
find_map 1.16x 1.02x 0.98x 0.99x -
take 1.12x 1.09x 1.01x 1.00x -
slice 1.12x 1.09x 1.01x 0.97x -
chain 1.14x 1.04x 1.02x 1.02x The items here are the nb of Iterable arguments, not the total number of items in the base PyoIterator.
product 1.03x 1.01x 1.01x 0.99x The items here are the nb of Iterable arguments, not the total number of items in the base PyoIterator.
next 1.31x 1.45x 1.47x 1.47x The items here are the nb of calls to next in a loop.
once_with 1.75x 1.83x 1.84x 1.84x The items here are the nb of time we create a PyoIterator with once_with(), and then call next on it.
map_with 0.97x 0.98x 0.97x 0.99x Slight regressions -> we must reconstruct the tuple args for each call (func, self, others).
tail 1.20x 1.24x 1.23x 1.19x -
Some::iter 4.66x 5.28x 5.28x 5.20x Iter is directly created from new constructor, no getattr pattern.
Null::iter 5.87x 7.13x 7.07x 6.96x Iter is directly created from new constructor, no getattr pattern.
Err::iter 5.75x 6.95x 6.92x 6.89x Iter is directly created from new constructor, no getattr pattern.
Ok::iter 7.16x 8.62x 8.71x 8.67x It was calling self.ok() before internally.
PyoIterable::iter 1.16x 1.19x 1.18x 1.17x -
Iter::__init__ 1.82x 1.93x 1.93x 1.92x Also impact PyoMutableSequence::{extract_if, drain} and PyoReversible::rev
Reversible::rev 5.23x 28.99x 28.08x 25.88x For Range, Vec and Seq types. ABC's in itself is 2-3% faster on small iterators after Rust migration.
Seq::__init__ 1.29x 1.14x 1.01x 1.01x -
Range::__init__ 1.62x 1.61x 1.58x 1.54x -
Seq::concat 2.58x 1.78x 1.10x 1.02x -

  • Rust migration and logic optimization: PyoIterator::fold_star args/kwargs truthiness are now matched to check if they are actually needed, and each case passes an optimized function to itertools::reduce. Without both for example, the method is was 1.2x faster. Once migrated to Rust, this case is now 2.23x faster. With args, the relative improvement is of 2.08x. With args AND kwargs, 1.87x.

  • Rust migration and logic optimization: PyoIterator::map_juxt is now fully in Rust as a "real" Iterator instead of a Callable used on builtin::map. Performance gains on small funcs tuples, performance regressions on large funcs tuples => 1.16x faster for 1 func, 1.08x for 4 funcs, 1.02x faster for 16 funcs, and 0.95x slower for 64 funcs. Optimizing the latter is on the roadmap, but all considered, I doubt most ppl will be using map_juxt with more than 16 funcs, so It's acceptable in the meantime.

  • Rust migration: PyoIterator::once is now 1.26x faster.

  • Logic optimization: PyoIterator::chain doesn't call from_iterable needlessly anymore. Unchanged on small iterators, but around 1.06x faster on 1k and 10k items when provided with a single Iterable argument.

  • PyoSequence::{first, last, get} are now around 1.7x (Range) to 2.1x (Seq) faster, after having both the ABC and the concrete classes migrated to Rust.

⚠️ Performance regressions

  • PyoIterator::{all, any} with no predicate is now fully in Rust. unfortunately, a constant ~100 ns performance regression has been observed on benchmarks. On a relative basis, this is a 7 to 8% performance hit on small iterators(10 to 100 items). On 1000 items, a 2% hit, and on 10k it's barely noticeable (absolute time is 35 us for reference on 10k items).

🛠️ Other improvements

  • Various minor docstring improvements.
  • Code readability and privacy management improvements in Rust source code.
  • PyoIterator::find has been moved to Rust, performance is identical as before.

🔗 Dependencies

  • Removed pytest-cov from dev dependencies.

[0.26.0] - 2026-06-16

💥 Breaking changes

  • API change: Inspect mixin is renamed to Tap, and it's inspect method is renamed to tap. This aligns better with the Rust crate tap and it's traits, as well as avoiding confusion with the inspect method on Result and Option types.
  • API change: Into mixin is renamed to Pipe, and it's into method is renamed to pipe. Same reasoning as for Inspect mixin, and also aligns with polars, pandas and sqlglot API's.
  • API change: Pipeable mixin is renamed to Fluent, to avoid confusion with Pipe mixin.
  • API change: SliceView::base property is renamed to inner.
  • Return type: PyoIterator::with_position returned Position values are now back to typing::Literal values, instead of enum::StrEnum instances. Unfortunately, Pyo3 doesn't have good native support for python enums yet, and this would have required a lot of maintenance burden for something that doesn't really change the behavior of the method.

🚀 Performance improvements

  • Iter::__bool__ is now 1.16x faster by avoiding to use itertools::islice, this logic has been replaced by a next call and sentinel object check.
  • PyoIterator::with_position has been moved to Rust. 2.88x faster.

✨ Enhancements

  • typing: Widened the input types of various methods on set-like concrete classes (Set, SetMut, StableSet, views classes).

🛠️ Other improvements

  • Fix: the check_nav script wasn't launching its main function.
  • Refactor: Use the tap crate troughout the Rust code for improved readability.
  • Refactor: replaced unwrap calls with expect or match cases in tools.rs for improved safety.
  • Various minor docstring improvements

🔗 Dependencies

  • Added tap crate for Rust code.

📦 Build system

  • Bumped Pyo3 version to 0.29.0.

[0.25.0] - 2026-06-09

🏆 Highlights

This release main change is a refactor of Iter and PyoIterator methods -> almost all methods from Iter have been moved to PyoIterator.

Only a handful of them remain in Iter, mostly instance methods relying on direct manipulation of the inner iterator, as well as the dunder overrides for concrete implemetnation.

💥 Breaking changes

  • API change: Iter::filter_star does not accept arbitrary Iterable as elements anymore, only tuples.
  • API change: The default Seq collector for Iter::collect has been deleted. All .collect() becomes .collect(Seq) if you want to keep the same behavior as before.

🚀 Performance improvements

  • no-copy: Avoid unecessary copy in Vec::repeat by constructing the new instance by reference.
  • Iter::filter_star has been moved to Rust. 1.45x to 1.50x faster across benchmarks (tuples with 2, 3, 4, 5 elements).
  • PyoIterator::{all, any} with predicate function migrated to Rust.1.02x to 1.05x faster.

✨ Enhancements

  • typing: Improved the input types of various PyoSet methods to align them with typeshed, as they were too strict.
  • typing: Iter::{map_star, filter_star} can handle 1 element tuples now.
  • typing: PyoIterable::iter now returns a PyoIterator instead of an Iter, allowing more flexibility for custom overrides.

🛠️ Other improvements

  • Splitted check_nav and generate_docs logic in two separate scripts. Closes issue #15.

[0.24.0] - 2026-05-30

🏆 Highlights

Iter::from_fn in Rust

This release improve the performance of Iter::from_fn by:

  • 1.35x with NO args/kwargs (i.e f() -> Option[T])
  • 1.5x with args OR kwargs
  • 1.6x with args AND kwargs

This means that you can define inline generators/iterators in a new way, without much, if any, performance trade-off in respect to the other three python-native paths.

In the benchmarks, only generator comprehensions has a slight performance advantage over Iter::from_fn.

This is fine, because comprehensions should be the go-to solution whenever possible, as they are concise, idiomatic/pythonic, and fast.

Performance vs python-native paths

The presence or absence of *args and **kwargs didn't have any substantial impact on the relative performance.

Python way Relative performance
Iterator class with __iter__ and __next__ Identical
function/method with yield statements Identical
generator comprehension, i.e (x for x in iterable) 0.9 to 0.95x (i.e from_fn is 5-10% slower)

💥 Breaking changes

  • Removed: PyoMutableSequence::{into_iter, extend_move}. What led to this decision is the fact that Python nature made these methods very situational, as well as their poor speed. into_iter can be replaced by iter, and x.extend_move(y) by x.extend(y) followed by y.clear().

🚀 Performance improvements

  • PyoMutableSequence::truncate is now 11x faster. The old implementation used an inefficient loop with pop() calls, while the new one uses del self[length:], after double-checking that this is a no-copy operation.
  • PyoMutableSequence::drain logic has been improved to avoid pop calls, which made it 6.5x faster. The subsequent Rust migration upped the improvement to 10.69x.
  • PyoMutableSequence::extract_if is the same story as drain. 5.18x improvement from a logic change, upped to 8.28x once ported to Rust.
  • Iter::successors migrated to Rust. 1.3x faster.

🔗 Dependencies

  • Dev: Deleted unused script dependencies: typer, polars.

🛠️ Other improvements

  • Refactor: Minor readability improvements in scripts/.

[0.23.1] - 2026-28-05

🐞 Bug fixes

  • typing: PyoIterator::sum has been giving false positives for a while. This is now fixed, with the same inputs/outputs as the python builtin.

🛠️ Other improvements

  • Internal: Python dev version is by default 3.13 to avoid any unsupported patterns on our minimum supported version.
  • Imports: Prioritizing typing::TYPE_CHECKING blocks import whenever possible. This could maybe speed-up initial import time.

[0.23.0] - 2026-28-05

🏆 Highlights

  • Feat: collections, new module mimicking python collections module, aiming to provide additional, more specialized collections types.

  • Feat: collections::StableSet, a mutable collection of unique elements which remember their insertion order. Uses dict::fromkeys internally, thus has the same characteristics as a dict regarding lookup/insertion/deletion/iteration performance.

  • Feat: collections::Deque, pyochain version of python builtin collections::deque.

  • Feat: SliceView, a zero-copy, composable slice view over any collections.abc.Sequence. It allows to create views into existing sequences without copying the data, and to compose them together in O(1) time. It also has an advance method for shifting the view's window forward or backward in-place, which can be useful for sliding windows. Credits to @eirikurt for the implementation and the idea, as well as@hwelch-fle for the typing improvements work, which I used as base for the integration in pyochain.

🆕 New features

  • Iter::map_with for mapping multiple iterables at once, just like map builtin when provided with multiple iterables.
  • Improved __eq__ for Set, SetMut, and Dict, with similar behavior than their internal data structures, as well as pyochain objects handling.
  • Set::__hash__, with similar behavior than it's internal frozenset structure
  • repeat method for Seq and Vec, which call the __mul__ dunder method.
  • Vec::repeat_mut, equivalent to my_list *= n.
  • Iter::from_repeat for repeating an object n times as elements of an Iter.
  • Dict::union_mut, similar to dic_1 |= dic_2 for in-place union of two dicts.
  • Vec::into_iter for consuming a Vec into an Iter that yield and pop each element from the Vec.
  • abc::{Into, Inspect} mixins, each providing one method from the original Pipeable. The latter inherit from both of them, thus no behavior change.
  • abc::{PyoMutableSet, PyoSized, PyoReversible, PyoContainer} ABCs. This don't impact runtime nor existing API, but is used for the ABC hierarchy tree to better mimick python collections.abc.
  • PyoSet::is_superset_strict was missing. Added.
  • Dict::from_keys, equivalent to dict.fromkeys python builtin.

💥 Breaking changes

  • Vec MRO: Vec does not inherit from Seq anymore, only from PyoMutableSequence. This means that if you were relying on Vec being a subclass of Seq for type checking or isinstance checks, you should now check against PyoMutableSequence instead.
  • API change: PyoIterator::{argmin, argmax} have been renamed to arg_min and arg_max, and their key argument version is now in the form of a separate method, arg_min_by and arg_max_by. Just like the last releases with similar changes, adapt your code in consequence.
  • Removed: PyoCollection::repeat has been removed. You can do my_collection.into(Iter.from_repeat, n) to get the exact same behavior.
  • API change: Dict::merge has been renamed to union.
  • API change: PyoIterable::{all, any, join, sum, min, my_by, max, max_by, all_unique_by, unpack_into} are moved to PyoIterator. Simply add a call to iter() in impacted code if a PyoCollection was used.
  • API change: PyoIterable::length has been removed. PyoIterator::count and PyoSized::len are their replacement. Closer to Rust semantics, and do things more explicitely, as the former is a full iteration that count the elements, while the latter is a call to len().
  • API change: All the methods that have been moved from PyoIterator to Iter in the 0.20.0 release are now back in their original ABC. To handle this, a new _from_iterable private method has been added, the idea being identical to what python stdlib does with set ABCs. See the method documentation for more details.
  • API change: Iter::with_position now yield Position(StrEnum) values instead of literal strings.
  • Removed: PyoIterable::all_unique has been removed and is now only on PyoIterator. If you used it on a PyoCollection, you can either add a call to iter() before, or if you want to keep the exact same underlying implementation, compare the length of the collection with the length of a Set created from it. Examples and explanations in the documentation of all_unique method.
  • Removed: PyoIterable::second. Use 2 calls to itertator.next() or sequence[1] instead.

🚀 Performance improvements

  • Moved to 🦀 -> Iter::filter_map. 1.15x (64 elements) to 1.25x (256, 1024, 4096 elements) faster.
  • Moved to 🦀 -> Iter::filter_map_star. 1.24x (64 elements) to 1.38x (256, 1024, 4096 elements) faster.
  • Moved to 🦀 -> Iter::scan. More or less 1.35x faster across sizes (64, 256, 1024, 4096, 16384 elements).
  • Moved to 🦀 -> Iter::map_while. 1.38x to 1.44x faster across sizes.
  • tail no-copy: PyoIterator::tail created internally a deque who was then re-wrapped in a Seq for the return. It now directly return a pyochain Deque created by reference from the aforementionned deque.

✨ Enhancements

  • typing: Iter::{filter, filter_false} now handle type narrowing for T | None and TypeGuard when possible.
  • API: Iter::{filter, filter_false} provided closure is now optional, just like python builtins.
  • API: Iter::{partition, collect} has been moved to PyoIterator, meaning all user-defined iterators can now call them.
  • API: Iter::from_fn now accepts *args and **kwargs to be passed to the generator function. Inputs are matched to minimize overhead and keep original performance if no additional arguments are needed.

📖 Documentation

  • Correctly handle hyperlinks in See Also sections. WIP to do the same for the rest of the sections.
  • Various docstrings changes with better explanations and examples.

📦 Build system

🛠️ Other improvements

  • Added --preview option to Ruff format workflow to garantuee markdown formatting
  • Reorganized the abc submodule by splitting the monolithic _iterable.py file into multiple ones with more specific content.
  • Moveed the views objects to a dedicated file for readability.
  • Improved the doc generator script redability and it's handling of submodules.

[0.22.0] - 2026-05-21

💥 Breaking changes

  • API: Vec::sort is split into Vec::sort and Vec::sort_by. If you were using Vec::sort(key=...), you should now use Vec::sort_by(key=...) instead. This aligns it with PyoIterator::{sort, sort_by} change from last release.

✨ Enhancements

  • Feat: Added a clearer repr for Range that shows the start, stop and step values.
  • Feat: Vec have an __eq__ method. expect same results than list::__eq__, except that it also works with other Vec instances.
  • Feat: Seq also now have an __eq__ method. Same story as Vec. It also has an __hash__ method, which is the same as tuple::__hash__, and is compatible with it, meaning that a Seq and a tuple with the same content will have the same hash.

🔄 Refactors

  • Less code duplication in scripts with a common module

🛠️ Other improvements

  • New pytest plugin improving docstring/markdown testing
  • Separated Ruff config in a dedicated toml file

📖 Documentation

  • Various improvements in docstrings, regarding formatting for the website, additional examples, etc..

[0.21.0] - 2026-05-21

💥 Breaking changes

  • Removed: Set::r_intersection, r_union, r_difference, r_symmetric_difference -> these methods were redundant and not really useful in practice, as the same behavior can be achieved by swapping the order of the sets in the non-reverse methods. For example, a.r_intersection(b) is equivalent to b.intersection(a), and so on for the other methods.
  • Migrated: PyoKeysView, PyoValuesView and PyoItemsView are moved from abc submodule to surface-level API. If you previously imported them as from pyochain.abc, update your imports to from pyochain instead.

🐞 Bug fixes

  • PyoKeysView and PyoItemsView lost their set-like methods union, intersection, etc... in the last release. It was an overlook of the removal of these methods from the PyoSet ABC. This is now fixed.

✨ Enhancements

  • Better returns: Due to the aforementioned bugfix, I took the opportunity to double check what was really returned from those operations before being wrapped in a self::__class__ call. Turns out that by default, keys and item views were returning set objects, which were immediately re-wrapped in their callers, which may have entailed unecessary copies, but most importantly the API loss of an already constructed object with more methods (SetMut). Now, each class (PyoKeysView, PyoItemsView, Set, SetMut) has a dedicated implementation of these methods, which garantee no-copy behavior with a better API.

[0.20.0] - 2026-05-21

💥 Breaking changes

  • API change: pyochain::traits module is now pyochain::abc. If you were importing from pyochain.traits, update your imports to pyochain.abc instead.
  • Removed: PyoIterable::new. Call __init__(()) for the same behavior, e.g Seq(), Iter(), etc...
  • Removed: Unzipped and Peekable dataclasses. The Iter methods who constructed them now simply return tuples instead, simplifying the API and improving speed.
  • API change: Iter::sort (now in base class PyoIterator) has been split into Iter::sort and Iter::sort_by. If you were using Iter::sort(key=...), you should now use Iter::sort_by(key=...) instead. This should bring typing improvements as well as a clearer API.
  • API change: PyoIterable::__init__ is deleted. This means that subclasses are free to implement their own constructors, without typing constraints nor default behavior.

Methods migration to concrete parents

If you did not define custom classes from PyoSet or PyoIterator, skip to the Enhancements section.

The __init__ deletion from PyoIterable also means that all ABCs methods that relied on self::__class__ needed to move to their concrete pyochain parents, as they couldn't stay (nor should have ever been) purely abstract.

This concerns:

from PyoSet to Set ->

intersection, r_intersection, union, r_union, difference, r_difference, symmetric_difference, r_symmetric_difference

from PyoIterator to Iter ->

take_while, skip_while, compress, unique, unique_by, take, skip, step_by, slice, cycle, insert, intersperse, chain, accumulate

✨ Enhancements

  • Migrated: Iter::{collect_into, try_collect, sort, tail} have been moved to PyoIterator, meaning all user-defined subclasses can now call them.
  • Migrated: Vec::{drain, extract_if} have been moved to PyoMutableSequence, meaning all user-defined mutable sequence subclasses can now call them.

📖 Documentation

  • Improved various classes and methods documentation.
  • Reformatted all code examples with Ruff.
  • Added dev documentation regarding design choices on where to implement a method (abstract vs concrete class).

🔄 Refactors

  • Separated some classes in dedicated files to reduce the size of _iter.py and improve readability. Seq is now in _seq.py, Set and SetMut are now in _set.py notably.

[0.19.0] - 2026-05-20

🏆 Highlights

  • dependency-free: pyochain is now dependency free and do not need cytoolz anymore. All the methods that were using it have been reimplemented in Rust or removed. See the breaking changes and performance improvements/regressions sections for more details.

💥 Breaking changes

  • API change: Iter::try_for_each now returns Result[tuple[()], E] instead of Result[None, E].
  • API change: PyoIterator::unique is split into Iter::unique and Iter::unique_by. If you were using Iter::unique(key=...), you should now use Iter::unique_by(key=...) instead.

Just like last release, some methods have beeen removed for a leaner, simpler API and a quicker no-dependency migration path.

See the table below for the removed methods and their recommended alternatives if you need the same behavior:

Method name Equivalent Notes
Iter::diff_at cytoolz::itertoolz::diff -
Iter::is_strictly_n more_itertools::strictly_n -
Iter::top_n cytoolz::itertoolz::topk -
PyoIterator::random_sample cytoolz::itertoolz::random_sample -
PyoIterator::interleave cytoolz::itertoolz::interleave Unpack self and the other iterables in a single one before calling the function.

🆕 New features

  • Vec::concat_mut for in-place concatenation of another Vec or list
  • Seq::concat for concatenation of another Seq or tuple.

🚀 Performance improvements

  • PyoMutableSequence::extend_move doesn't use functools::partial internally anymore. Expect some very light performance improvements.

Rust migrations🦀

Various methods have been migrated to Rust, from Python or Cython.

See the table below for the performance (2x means 2 times faster, 0.5x means 2 times slower).

Unfortunately, 3 methods have seen a performance regression vs the old Cython implementation, see the next section for details.

Method name From Improvement Notes
Iter::try_for_each Python 4.6 - 4.7x -
Iter::try_collect Python 2.3 - 3x -
PyoMutableSequence::retain Python 1.35 - 1.4x -
Iter::{map_windows, map_windows_star} Cython n=32: 1.17x, n=128: 1.40x Slower for smaller window sizes (0.81x for n=2, 0.93x for n=8).
Iter::map_juxt Cython 1.2x to 1.5x Slower for a single func (0.95x), but not useful in practice, use Iter::map instead

⚠️ Performance regressions

The 3 following methods have been migrated from Cython to Rust, and have unfortunately seen a performance regression compared to their old Cython counterparts.

However, they are still much faster than a pure Python implementation.

See the table below for the details.

Method name Vs Cython VS Python Notes
PyoIterator::unique_by 0.95 2.31x Equivalent to old PyoIterator::unique(key=...)
PyoIterator::unique 0.75 7.3X -
PyoIterator::intersperse 0.60 4.95x -

✨ Enhancements

  • typing: Some(NONE) is directly inferred as Option[T], lowering the number of potential errors.
  • typing: Iter::try_collect has received new overloads and should be inferred more accurately now.
  • typing: Iter::map_juxt has received new overloads to precisely infer the return type up to 10 functions.

🐞 Bug fixes

  • PyoIterator::{last, length} were not properly handling custom Iterators and StopIteration errors. Fixed.

🔄 Refactors

  • Renamed converters.rs file in Rust to mixins.rs to better reflect its content and purpose.
  • Internal: Renamed PyOk and PyErr to PyoOk and PyoErr to avoid any confusion with PyO3 types.
  • Internal: Various renaming and cosmetic changes in Rust for better readability and consistency.

📖 Documentation

  • Various reorganisation changes to reduce redundancy
  • Updated CONTRIBUTING.md with latest architecture status
  • Improved docstrings of various classes and methods with better examples and explanations
  • Fix: PyoIterator::for_each_star was wrongly typed as working with an Iterator containing any Iterable elements, instead of only tuples.

🧪 Tests

  • standardized iteration sizes across benchmarks over iterators/collections

🛠️ Other improvements

  • Various internal improvements to scripts for code readability
  • Added checks in scripts/check_doscstrings to flag invalid existing links in zensical navigation, README and CONTRIBUTING.md, to prevent future broken links from being merged.

🔗 Dependencies

  • Added sdsort as a dev dependency for code formatting and sorting.

[0.18.0] - 2026-05-18

💥 Breaking changes

  • Removed: PyoIterator::is_distinct was doing the exact same thing as PyoIterator::all_unique without a key function.
  • Removed: Iter::{partition_all, partition_by}. Call the corresponding functions in cytoolz.itertoolz instead.
  • Removed: Iter::unique_to_each. It was acting like it was lazy, but it was immediately consuming the whole iterator and storing the results in memory, as well as being niche. If you need this behavior, use the more_itertool corresponding function instead.
  • Removed: PyoIterator::elements -> this was not lazy, but the API acted like it was. If you need this behavior, call collections::Counter::elements instead.
  • Removed: Vec::most_common -> simple wrapper around collections::Counter::most_common. Call it directly instead if you need it.
  • Removed: Iter::{split_before, split_after, split_at, split_into, split_when}. Call the corresponding functions in more_itertools instead.
  • Behavior change: PyoIterator::partition now return a tuple of Vec from a predicate function, instead of an Iter of tuples of length n. It aligns with the Rust implementation, and is often more useful in practice. Call the corresponding function in cytoolz.itertoolz instead if you need the old behavior.
  • API change: PyoIterator::all_unique does not accept a key function anymore. If you need one, call all_unique_by instead.

✨ Enhancements

  • Check safety: Guarantee singleton behavior for Null. Calling Null() will always return the same instance, which is NONE. This allows for identity checks (is) to work as expected with Null, and ensures that there are no multiple instances of Null floating around in the system. This also means that you can use Null() instead of NONE if you prefer, without worrying about breaking the singleton property.
  • Feat: PyoIterator::all_unique is migrated to PyoIterable, meaning ALL collections now can call it without converting to an iterator first. PyoSequence and PyoSet (and by extension Seq, Set, etc...) have their own optimized implementations
  • Feat: Added PyoIterable::all_unique_by for checking uniqueness based on a custom key function. This is the same as former all_unique(key=...), but with a clearer name and intent.

🚀 Performance improvements

  • Using cast_exact/is_exact_instance_of instead of cast/is_instance_of when interacting with pyochain types (Result/Option) in Rust methods bring an overall +2% to +5% performance gain across benchmarks. Various Option/Result methods, as well a Iter::{try_reduce, try_find, try_fold}, benefit from this change.
  • all_unique now call cython level code and is has fast as the deprecated is_distinct method.
  • PyoIterable::all_unique_by (old PyoIterator::all_unique(key=...) path) is now in Rust, and is 30 (100 elements) to 70% (500 elements) faster than the old implementation. Larger collections should see even better improvements.
  • PyoIterable::all_unique is in Rust as well, with a slight performance regression of around 1 to 5% compared to the old is_distinct method. Note that if you were using all_equal instead, you should expect performance improvement instead, in the same ballpark as all_unique_by.

🔄 Refactors

  • Renamed PyNone to PyNull in Rust to avoid confusion with Pyo3 types.
  • Various cosmetic changes in Rust to improve readability and documentation
  • Extracted PyoIterable::{first, second}, PyoIterator::{chain, insert, step_by}, Iter::tail from cytoolz to python, as they were trivial function calls who didn't needed any external dependency work.
  • Extracted PyoIterable::{last, length} from cytoolz to Rust. No performance regression.

🐞 Bug fixes

  • Replaced cytoolz calls in Iter::skip. Slight performance regression, but cytoolz wasn't creating a new Iterator, but was eagerly consuming the original one until n elements were skipped, which could be a problem with large n/collections, and could cause issues if lazyness was expected. The new implementation uses itertools.islice, which creates a new iterator that skips the first n elements without consuming the original one.

🛠️ Other improvements

  • Benchmarks -> deletions, renaming and new ones
  • Cleaned up unneded/redundants tests for args concatenation and Result
  • Docstring checker was broken and not flagging uncorrect docstrings. Fixed.

📖 Documentation

  • Small improvements in Iter and PyoIterable docstrings

[0.17.0] - 2026-05-17

🏆 Highlights

Option redesign for 100% type safety

The Option type has been redesigned to be more consistent with Result, with a new "dummy" Protocol OptionType as the base class for Some and NONE.

This change allows pattern matching exhaustiveness checks and much simpler internal Rust implementation, since now Option is, just like Result, a type union from the POV of type checkers. In Rust, it's an empty struct. No performance impact has been observed.

💥 Breaking changes

Option redesign

The following methods have been removed or renamed:

  • Option::__init__ => option (pure function)
  • Option::if_true => then_if_true (pure function)
  • Option::if_some => then_if_some (pure function)

📖 Documentation

  • New template sections in CONTRIBUTING.md for standardizing the CHANGELOG and releases

🔄 Refactors

  • Caller sites of Option::__init__ have been updated

🛠️ Other improvements

  • Various new benchmarks to cover impacted methods from the breaking changes

[0.16.0] - 2026-05-17

✨ Enhancements

  • Iter::for_each_star now handle args and kwargs in the same way as Iter::for_each, allowing to pass arguments to the function being called for each item.

🚀 Performance improvements

  • Migrated Iter::for_each, for_each_star to Rust. 1.5x-2x faster in average.
  • Range did not have __slots__ properly set. Fixed.

🔄 Refactors

  • New traits in rust::types to improve the readability when handling Callable arguments with args and kwargs. instead of call(func, self, args, kwargs), we can now do func.call(self, args, kwargs).
  • Internal scripts cosmetic changes to standardize them with recommended pattern -> from pyochain import x instead of import pyochain as pc

📖 Documentation

  • Improvements to traits::PyoSet methods documentation, with better examples and explanations.
  • Updated documentation to reflect the new pattern -> from pyochain import x instead of import pyochain as pc
  • Various other minor improvements and fixes.

🛠️ Other improvements

  • pytest-benchmark setup, used and now part of the developpement process to catch performance regressions.
  • Various other minor improvements and fixes.