audio of screensharing on ios extremely deep

[DuyTruong123456]

Hi guys , I'm have a problem with webrtc ios sdk . When I pass the CMSampleBuffer that i got from ios screenbroadcast extension , the sound were changed to very deep voice . What could have been the possible cause ?

[Mohit-3196]

Hi @DuyTruong123456,
How are you?

Thank you for your question. Let me invite my colleague @mekya to help us with this.

Also, it would be great if you could define the reproduce scenario for the issue or share some samples.

Short description

Brief description of what happened

Environment

• Operating system and version:
• Java version:
• Ant Media Server version:
• Browser name and version:

Steps to reproduce

Expected behavior

Put as much detail here as possible

Actual behavior

Put as much detail here as possible

Thank you,
Mohit

[DuyTruong123456]

@Mohit-3196
Short description
Brief description of what happened

• i tried to use web-rtc-ios-sdk to add screenbroadcast audio for ios
  Steps to reproduce

• Got audio CMSampleBuffer from ios screensharing extension

• Convert buffer to data

• use socket connection to write data

• get data out in main project

• convert to CMSampleBuffer using below code :

• (void)processAudioData:(NSData *)data {
  CMBlockBufferRef blockBuffer = NULL;
  OSStatus status = CMBlockBufferCreateWithMemoryBlock(
  kCFAllocatorDefault,
  (void *)data.bytes,
  data.length,
  kCFAllocatorNull,
  NULL,
  0,
  data.length,
  0,
  &blockBuffer
  );

  if (status != kCMBlockBufferNoErr) {
  NSLog(@"CMBlockBuffer creation failed with status: %d", (int)status);
  return;
  }
  [self printFullData:data];
  // Create an audio format description.
  // This is an example format; adjust as necessary for your actual audio format.
  AudioStreamBasicDescription asbd = {0};
  sbd.mSampleRate = 43000;
  asbd.mFormatID = kAudioFormatLinearPCM; // 'lpcm' for linear PCM
  asbd.mFormatFlags = kAudioFormatFlagIsBigEndian | kAudioFormatFlagIsSignedInteger | kAudioFormatFlagIsPacked;
  asbd.mBytesPerPacket = 2;
  asbd.mFramesPerPacket = 1;
  asbd.mBytesPerFrame = 2;
  asbd.mChannelsPerFrame = 1;
  asbd.mBitsPerChannel = 16;
  asbd.mReserved = 0;

  CMAudioFormatDescriptionRef audioFormatDescription = NULL;
  status = CMAudioFormatDescriptionCreate(
  kCFAllocatorDefault,
  &asbd,
  0,
  NULL,
  0,
  NULL,
  NULL,
  &audioFormatDescription
  );

  if (status != noErr) {
  NSLog(@"CMAudioFormatDescription creation failed with status: %d", (int)status);
  CFRelease(blockBuffer);
  return;
  }

  CMSampleBufferRef sampleBuffer = NULL;
  const size_t sampleSizeArray[] = { data.length };
  status = CMSampleBufferCreate(
  kCFAllocatorDefault,
  blockBuffer,
  true,
  NULL,
  NULL,
  audioFormatDescription,
  data.length / asbd.mBytesPerFrame,
  0,
  NULL,
  1,
  sampleSizeArray,
  &sampleBuffer
  );

  if (status != noErr) {

    NSLog(@"CMSampleBuffer creation failed with status: %d", (int)status);
    CFRelease(blockBuffer);
    CFRelease(audioFormatDescription);
    return;
  

  }

  // Process the sample buffer as needed
  [self handleSampleBuffer:sampleBuffer];

  // Clean up
  CFRelease(sampleBuffer);
  CFRelease(blockBuffer);
  CFRelease(audioFormatDescription);
  }
  -please note that i have check the AudioStreamBasicDescription in CMSampleBuffer i got from ios extension , and tried to set it to the code but it result in undefined noise , i tried to adjust it and sound get better but still extremely deep.

• then i init my peer with this method :

• (instancetype)initWithEncoderFactory:(nullable id)encoderFactory
  decoderFactory:(nullable id)decoderFactory
  audioDeviceModule:(RTCAudioDeviceModule *)audioDeviceModule NS_AVAILABLE_IOS(2_0);
  Expected behavior

• The sound broadcasting is correct
  Actual behavior

• Undefined noise if use same AudioStreamBasicDescription as i checked in extension and extremely deep nosie if i tried to adjust it

[mekya]

Hi @DuyTruong123456,

I'm curious what is the reason to not use the sample we provide for screen sharing. It clearly captures the screen audio and publish to the Ant Media Server

https://github.com/ant-media/WebRTC-iOS-SDK/tree/master/ScreenShare

Let me know if I can help you further

Cheers