How can I record from two devices at the same time and write it to one encoder?

[m1maker]

Hello.
I have a question. I have a callback function that can write to ma_encoder from a device, and everything works when I set the callback to one ma_device.
But I want to make it possible to record from a microphone and also use a loopback device.
And it works, but when listening to the file I hear crackling and sound slowdowns.
What I didn’t do: I tried setting sizeInFraims, I tried lowering SampleRate, I even launched each device in a separate thread, but it didn’t work.
Please tell me if there is a solution to this problem?

[mackron]

You need to mix them in into an intermediary buffer first, and then write out that buffer to the encoder. Mixing is just summing the samples together.

[m1maker]

Ok. To understand, void* pInput is a pointer to what it is

[m1maker]

Now everything is mixed, the main device doesn’t even wheeze. However, there are some problems with the loopback device. It doesn't wheeze, but it still doesn't sound the way it should sound.
float* mix(float* input1, float* input2, ma_uint32 frameCount) {
float* result = new float[frameCount * 2];
for (ma_uint32 i = 0; i < frameCount * 2; i++) {
result[i] = (input1[i] + input2[i]) / 2.0f;
}
return result;
}

float* loopback_buffer = nullptr;
void audio_recorder_callback(ma_device* pDevice, void* pOutput, const void* pInput, ma_uint32 frameCount)
{
ma_encoder* encoder = (ma_encoder*)pDevice->pUserData;
if (g_CurrentDevice.loopback == L_NOT_USED)
ma_encoder_write_pcm_frames(encoder, pInput, frameCount, nullptr);
else {
if (loopback_buffer != nullptr) {
float* result = mix((float*)pInput, loopback_buffer, frameCount);
ma_encoder_write_pcm_frames(encoder, result, frameCount, nullptr);
}
}
(void)pOutput;
}
void audio_recorder_callback_loopback(ma_device* pDevice, void* pOutput, const void* pInput, ma_uint32 frameCount) {
loopback_buffer = (float*)pInput;
}

[mackron]

Does the loopback device record properly in isolation without the microphone? I would first check that you're using the same format/channel/rate in your two device configs so that they're consistent with each other.

Also, unrelated to any of your glitches, when mixing, you don't need to divide by 2. That will be halving your volume.

[m1maker]

Yes. I'd try to force set parameters, but no result.

class audio_recorder {
public:
	ma_device_config deviceConfig;
	ma_device_config loopbackDeviceConfig;
	ma_encoder_config encoderConfig;
	ma_encoder encoder;
	ma_device recording_device;
	ma_device loopback_device;
	audio_recorder() {
		CreateDirectory(L"recordings", nullptr);
		std::string file = "recordings/" + get_now() + ".wav";
		encoderConfig = ma_encoder_config_init(ma_encoding_format_wav, ma_format_f32, 1, 48000);
		ma_encoder_init_file(file.c_str(), &encoderConfig, &encoder);
		deviceConfig = ma_device_config_init(ma_device_type_capture);
		if (!g_CurrentInputDevice.default)
			deviceConfig.capture.pDeviceID = &g_CurrentInputDevice.id;
		deviceConfig.capture.format = ma_format_f32;
		deviceConfig.capture.channels = 1;
		deviceConfig.sampleRate = 48000;
		deviceConfig.periodSizeInFrames = 256;
		deviceConfig.dataCallback = audio_recorder_callback;
		deviceConfig.pUserData = &encoder;
		if (g_CurrentOutputDevice.active) {
			loopbackDeviceConfig = ma_device_config_init(ma_device_type_loopback);
			loopbackDeviceConfig.capture.pDeviceID = &g_CurrentOutputDevice.id;
			loopbackDeviceConfig.capture.format = ma_format_f32;
			loopbackDeviceConfig.capture.channels = 1;
			loopbackDeviceConfig.sampleRate = 48000;
			loopbackDeviceConfig.periodSizeInFrames = 256;
			loopbackDeviceConfig.dataCallback = audio_recorder_callback_loopback;
			loopbackDeviceConfig.pUserData = nullptr;

[mackron]

To help you isolate the issue my recommendation is to get the loopback device working by itself first without the microphone. Then incorporate the microphone ones that's working. One thing to experiment with is increasing the size of periodSizeInFrames from 256 to 512 or 1024 and see how changes things.

[m1maker]

Ok. Thank you. I stopped in 4096.
I think this is the last question. How can I do Resampling?
That is, I have a device with a purity of 48000, and it cannot be changed. How can I resample this and record it to 44100 for example?

[mackron]

You should rarely need to do resampling yourself. If you wanted to have everything be 44100, just set it via the device config and miniaudio will do any necessary resampling for you.

If for some reason you need to manually do resampling, you can use ma_resampler, but I would highly recommend you only use that as a last resort.