iOS app crash when _nativePlugin.createPipContentView()

[Sahilpervez]

[iOS Crash] App crashes when calling _setupPip() using pip: ^0.0.3 with agora_rtc

Description

I am implementing Picture-in-Picture (PiP) mode for the video call feature in my Flutter app.

• Using agora_rtc for the video call functionality.
• Using pip: ^0.0.3 to implement PiP.
• Everything works perfectly on Android.
• On iOS, the app crashes as soon as the line _pipContentView = await _nativePlugin.createPipContentView(); is executed.

How I'm using it

I found _setupPip() method in example section of the pip plugin. I call the _setupPip() function inside the initState() of the video call screen to enable PiP before the video call starts.

Function that causes the crash

Future<void> _setupPip() async {
  if (Platform.isIOS && _pipContentView == 0) {
    _pipContentView = await _nativePlugin.createPipContentView(); // crash happens here
    print('[createPipContentView]: $_pipContentView');
    _pipContentView = _pipContentView;
    if (mounted) {
      setState(() {});
    }
  }

  final options = PipOptions(
    autoEnterEnabled: _autoEnterEnabled,
    aspectRatioX: aspectRatioX.toInt(),
    aspectRatioY: aspectRatioY.toInt(),
    sourceRectHintLeft: 0,
    sourceRectHintTop: 0,
    sourceRectHintRight: 0,
    sourceRectHintBottom: 0,
    seamlessResizeEnabled: true,
    useExternalStateMonitor: true,
    externalStateMonitorInterval: 100,
    contentView: _pipContentView,
    sourceContentView: _playerView,
    preferredContentWidth: 200,
    preferredContentHeight: 200,
    controlStyle: 2,
  );

  try {
    final success = await _pip.setup(options);
    print('PiP Setup ${success ? 'successful' : 'failed'}');
  } catch (e) {
    print('PiP Setup error: $e');
  }
}

I have changed the version of sdk in native_plugin from (^3.7.0 to 3.6.2) to match the version of sdk that is used in my project.

environment:
  sdk: ^3.6.2
  flutter: '>=3.3.0'

flutter doctor log :

fvm flutter doctor

Doctor summary (to see all details, run flutter doctor -v):

[✔] Flutter (Channel stable, 3.27.4, on mac05 15.5 24F74 darwin-x64, locale en-IN)

[✔] Android toolchain - develop for Android devices (Android SDK version 35.0.0)

[✔] Xcode - develop for 105 and macOS (Xcode 16.4)

[✔] Chrome-develop for the web

[✔] Android Studio (version 2024.1)

[✔] VS Code (version 1.89.1)

[✔] Connected device (2 available)

[✔] Network resources

Please let me know If I need to do some extra setup in my_project/ios folder to get the things to work smoothly on iOS.

[Siddharth2212]

I am facing same issue. Any updates?

[Sahilpervez]

I am facing same issue. Any updates?

Not yet, I'm going through the waterbus codebase and waterbus SDK looking at how the have implemented it.
I found it here : #10 (comment)

[khlebobul]

Hello, I still have a problem on iOS, but in my case it's just a black screen in the PiP picture. Also, I don't work with agora_rtс.

Tell me, please:

• Info.plist contains UIBackgroundModes=audio?
• Is the native AVPictureInPictureController connected to agora_rtс?

UPDATE: App crashes (same issue with webrtc)

[hensi-elaunch]

I am facing same issue.
Any updates regarding how to show flutter video call view in ios pipview?

[peilinok]

@Siddharth2212 @khlebobul @Sahilpervez @hensi-elaunch

Hi all, for the official Picture-in-Picture (PiP) integration, check out the guide here: https://github.com/AgoraIO-Extensions/Agora-Flutter-SDK/blob/release/6.5/docs/integration/Picture-in-Picture.md

Let me know if you have any questions!

[Sahilpervez]

Hello, I still have a problem on iOS, but in my case it's just a black screen in the PiP picture. Also, I don't work with agora_rtс.

Tell me, please:

• Info.plist contains UIBackgroundModes=audio?
• Is the native AVPictureInPictureController connected to agora_rtс?

UPDATE: App crashes (same issue with webrtc)

yes in mine UIBackgroundModes is set to audio
I didn't find any options to connect it to agora_rtc, I tried with web_rtc but it didn't work for me.

[Sahilpervez]

@Siddharth2212 @khlebobul @Sahilpervez @hensi-elaunch

Hi all, for the official Picture-in-Picture (PiP) integration, check out the guide here: https://github.com/AgoraIO-Extensions/Agora-Flutter-SDK/blob/release/6.5/docs/integration/Picture-in-Picture.md

Let me know if you have any questions!

Thanks a lot for providing this,
I'll try it and let you know.

[sujithsureshkumar]

you are calling example code , thats why it is crash. In ios , you can only open ios native AVlayer in pip mode. So first render the view in ios & take the view memory or pointer to flutter , using UIview pointer you can pass the pointer in pip content.
This pip plugin is an indirect method for ios, because finding UIView() memory passing to flutter & pass the pointer back to ios need more effort. So you can simply initialise AVPictureInPictureController inside swift or objective c code, & add the Avlayer UIView to the controller , that's it . This is un easy way.

Note:
Only AVLayer works in pip, so first convert UIView to an AVLayer

you can use this plugin only for android

I just typed this info in fast mode, I will improve my explanation later.

[Ajay1598]

@Siddharth2212 @khlebobul @Sahilpervez @hensi-elaunch
Hi all, for the official Picture-in-Picture (PiP) integration, check out the guide here: https://github.com/AgoraIO-Extensions/Agora-Flutter-SDK/blob/release/6.5/docs/integration/Picture-in-Picture.md
Let me know if you have any questions!

Thanks a lot for providing this, I'll try it and let you know.

import 'package:agora_rtc_engine/agora_rtc_engine.dart';

// Declare controllers
late final RtcEngine _engine;
late final AgoraPipController _pipController;

// Create and initialize RtcEngine
_engine = createAgoraRtcEngine();
await _engine.initialize(RtcEngineContext(
// ... configuration
));

// Create PiP controller
_pipController = _engine.createPipController();

...AgoraPipController. is not define in agora_rtc_engine package ??