Deadlocks in tests after upgrading rxdart

[Mike278]

Took another stab at upgrading off rxdart 0.23.1, but unfortunately still getting some failing tests. Here's a repro/comparison (could probably be minimized further but not sure which parts are relevant).

import 'package:meta/meta.dart';
import 'package:moor/ffi.dart';
import 'package:moor/moor.dart';
import 'package:rxdart/rxdart.dart';
import 'package:test/test.dart';

class TestDb extends GeneratedDatabase {
  TestDb() : super(SqlTypeSystem.withDefaults(), VmDatabase.memory());
  @override final List<TableInfo> allTables = const [];
  @override final int schemaVersion = 1;
}

Future<void> testCase({
  @required Stream<int> Function() createOuter,
  @required Stream<int> Function() createInner,
}) async {
  final log = <int>[];
  final timeout = Duration(milliseconds: 100);

  final a = createOuter();
  final b = a.switchMap((_) => createInner());

  b.listen(log.add);
  await b.first.then(log.add)
    .timeout(timeout, onTimeout: () => fail('1st should complete'));
  expect(log, [2, 2]);

  b.listen(log.add);
  await b.first.then(log.add)
    .timeout(timeout, onTimeout: () => fail('2nd should complete'));
  expect(log, [2, 2, 2, 2]);
}

void main() {
  group('rxdart upgrade', () {
  
    test("moor", () async {
      final db = TestDb();
      Stream<int> selectInt(int i) => db
        .customSelect('select $i a')
        .map((row) => row.read<int>('a'))
        .watchSingle();
      await testCase(
        createOuter: () => selectInt(1),
        createInner: () => selectInt(2),
      );
    });

    test("rxdart", () async {
      final outer = BehaviorSubject<int>();
      final tc = testCase(
        createOuter: () => outer,
        createInner: () {
          final inner = BehaviorSubject<int>();
          Future.delayed(Duration(milliseconds: 10)).then((_) => inner.add(2));
          return inner;
        }
      );
      await Future.delayed(Duration(milliseconds: 10));
      outer.add(1);
      await tc;
    });
  });
}

• rxdart 0.23.1: Both tests pass
• rxdart 0.24.0 and 0.24.1: Both tests fail with "2nd should complete"
• rxdart 0.25.0, 0.26.0, and 0.27.0: Only the moor test fails
• All the tests were done with moor 4.3.1, but I also spot checked moor 3.4.0 + rxdart 0.27.0 and I get the same result.

Since this began after upgrading rxdart I've started by creating an issue here, but @simolus3 might be able to help determine if this is a moor bug.

#477 and #500 might have some context from the last time we tried to upgrade off rxdart 0.23.1.

I looked around for similar issues since last time and came #511 looks similar. The 1st repro in that issue still times out with rxdart 0.27.0, even though the 2nd repro looks fixed.

[simolus3]

I'm taking a look at this now. Interestingly both tests pass if you cancel the first subscription before the second run:

  final sub = b.listen(log.add);
  await b.first
      .then(log.add)
      .timeout(timeout, onTimeout: () => fail('1st should complete'));
  expect(log, [2, 2]);
  await sub.cancel();

[simolus3]

My take is that RxDart is doing some understandable but questionable things here. First, it looks like the root of the failure is from forwardingStream:

rxdart/lib/src/utils/forwarding_stream.dart

Lines 68 to 74 in 3975769

	} else if (stream.isBroadcast) {
	controller = StreamController<R>.broadcast(
	onListen: onListen,
	onCancel: onCancel,
	sync: true,
	);
	} else {

This function is used by most transformers from RxDart, including switchMap. It maps broadcast streams to broadcast streams (which is reasonable), but the broadcast stream controller will only invoke the onListen callback for the first listener. This means that the "inner" stream will only get listened to once! Since moor streams emit their cached value in the onListen callback (where else would they do it), you don't get a value on the second subscription.

The reason RxDart subjects don't suffer from the same behavior is because forwardingStream treats them specially. For example, we can use the following class to obtain a Stream that should behave exactly like a Subject wrt. to the Stream api in dart:async: Extracted to gist due to length.

When one only uses the stream api from dart:async, a subject should be indistinguishable from _SyncProxyStream(subject). However, this change causes the rxdart test to fail as well:

      final outer = BehaviorSubject<int>();
      final tc = testCase(
          createOuter: () => _SyncProxyStream(outer), // Test fails because of this
          createInner: () {
            final inner = BehaviorSubject<int>();
            Future.delayed(Duration(milliseconds: 10))
                .then((_) => inner.add(2));
            return inner;
          });
      await Future.delayed(Duration(milliseconds: 10));
      outer.add(1);
      await tc;

This means that RxDart's transformers don't reliably work on every Stream, which is really unfortunate. At the same time, I understand the need to keep "broadcastability" through stream transformers which probably wouldn't work otherwise.

My suggestion would be to make forwardingStream return a single-subscription stream when the input is not a Subject, but I'm not sure if there are negative consequences of that as well. It's certainly a breaking change.

[frankpepermans]

To understand the issue correctly,
do you expect that onListen fires on every new Subscription?

Because, that is not the case, for example:

import 'dart:async';

void main() {
  var count = 0;
  final controller = StreamController<int>.broadcast(onListen: () => print('listen to me! ${++count}'));
  
  controller.stream.listen((_) {});
  controller.stream.listen((_) {});
  controller.stream.listen((_) {});
  controller.stream.listen((_) {});
}

Here, the print invokes only once, "listen to me! 1".

As I recall, this was actually wrong in earlier rxdart versions, where onListen would actually do trigger on every subscription.

If you cancel them all, and then do a new subscription, then onListen invokes again.

[simolus3]

do you expect that onListen fires on every new Subscription?

In general, yes! I know that the onListen callback from StreamController.broadcast is not necessarily invoked for every subscription, but I don't think that RxDart should make that assumption for every broadcast stream.

For instance, consider this example:

class DoOnSubscribeStream<T> extends Stream<T> {
  final Stream<T> inner;
  final void Function() onSubscribe;

  DoOnSubscribeStream(this.inner, this.onSubscribe);

  @override
  bool get isBroadcast => inner.isBroadcast;

  @override
  StreamSubscription<T> listen(void Function(T event)? onData,
      {Function? onError, void Function()? onDone, bool? cancelOnError}) {
    onSubscribe();
    return inner.listen(onData,
        onError: onError, onDone: onDone, cancelOnError: cancelOnError);
  }
}

void main() {
  final controller = StreamController.broadcast();
  final stream = DoOnSubscribeStream(controller.stream, () {
    print('new subscription');
  });

  stream.listen(null); // prints!
  stream.listen(null); // prints!
  stream.listen(null); // prints!

  final switched = stream.switchMap((_) => Stream.empty());
  switched.listen(null); // prints!
  switched.listen(null); // does not print -- why?
}

I think it's perfectly reasonable to expect the last line to print subscribed as well.

In fact, there are less-contrived examples of this as well. For instance, let's say someone writes their own broadcast stream that emits a cached value for each new listener if one is available (moor is doing pretty much that). Those streams depend on new listens not being dropped by some transformer down the line.

In my opinion, the special case for subjects in forwardingStream shows that new subscriptions should propagate even for broadcast streams. If those weren't special cased, switchMap wouldn't properly work for behavior subjects for instance. I strongly feel that RxDart shouldn't give its own stream implementations any special treatment, it's fragile and breaks for transformations that may be implemented by third-party libraries.

[frankpepermans]

We only use forwardingStream for transformations that need to do something on special events, like onListen, onPause, onResume or onCancel.
Not all our transformers need it, some use a normal Stream sink.

But these events fire just like they would on "normal" Dart streams, and we follow the normal Stream behavior with rxdart.

[frankpepermans]

Ok I now get what you mean,

You invoke on the actual listen method, not on the onListen events,
in that case, yes we have a difference in rxdart indeed...

Will see how breaking that would be

[frankpepermans]

See here for a PR which would fix your issue: #588

[Mike278]

Thank you both for the quick responses! Glad to see it's a simple fix. I'll try to give that branch a try tomorrow.

[frankpepermans]

@Mike278 Sorry I was a bit too fast there, the fix would be a little more complex unfortunately, I'll keep you posted :/

[frankpepermans]

So that PR branch probably works for your listen issue, but we need a better solution of course, it's a bit hacky atm.

Do feel free to try it out in the meantime of course.

[hoc081098]

do you expect that onListen fires on every new Subscription?

In general, yes! I know that the onListen callback from StreamController.broadcast is not necessarily invoked for every subscription, but I don't think that RxDart should make that assumption for every broadcast stream.

RxDart's forwarding stream behavior is the same as fromHandlers (stream_transform) https://github.com/dart-lang/stream_transform/blob/533e1af154629959545e3a3728312af6cb4f5619/lib/src/from_handlers.dart#L10.
I think that is expected behavior.

[frankpepermans]

Yes, but the issue here is that they expect the listen handler to be invoked on each listen:

import 'dart:async';

import 'package:rxdart/rxdart.dart';

class DoOnSubscribeStream<T> extends Stream<T> {
  final Stream<T> inner;
  final void Function() onSubscribe;

  DoOnSubscribeStream(this.inner, this.onSubscribe);

  @override
  bool get isBroadcast => inner.isBroadcast;

  @override
  StreamSubscription<T> listen(void Function(T event)? onData,
      {Function? onError, void Function()? onDone, bool? cancelOnError}) {
    onSubscribe();
    return inner.listen(onData,
        onError: onError, onDone: onDone, cancelOnError: cancelOnError);
  }
}

void main() {
  final controller =
      StreamController<void>.broadcast(onListen: () => print('I start!'));
  final stream = DoOnSubscribeStream(controller.stream, () {
    print('new subscription');
  });

  var switched = stream.switchMap((event) => Stream.value(event));
  switched.listen(null); // prints!
  switched.listen(null); // does not print!

  var mapped = stream.map((event) => event);
  mapped.listen(null); // prints!
  mapped.listen(null); // prints!
}

it prints "I start!" once, as expected, but if you override the listen handler and expect it to run on each actual listen, then it is inconsistent.

[hoc081098]

IMO DoOnSubscribeStream means calling callback when controller.stream is listen to, because the Stream is broadcast, it should be listen once :))

[hoc081098]

Only built-in operators cause the inconsistent, but stream_transform package (from dart-lang org) does not follow that behavior 😕