Using FFmpeg instead of MP4Recorder.

[laricheveg]

//Record it
recorder.record(incomingStream);

Could you suggest
Is it possible / how to catch stream using ffmpeg instead of this recorder ?

Thanks!

[bakoushin]

You can instantiate a StreamerSession on a specific port and send RTP stream from FFmpeg to that port.

[laricheveg]

@bakoushin i need to receive webrtc by ffmpeg. So ffmpeg should receive stream from StreamerSession?

[laricheveg]

#4
this code samples doesn't work

[bakoushin]

Probably this code sample is not supposed to be fully working, it just explains how things could be configured.

I was able to run a H264 stream trough a Streamer using the following configuration:

const { MediaInfo, CodecInfo } = require('semantic-sdp');
const MediaServer = require('medooze-media-server');

const streamer = MediaServer.createStreamer();

const video = new MediaInfo('video', 'video');
video.addCodec(new CodecInfo('h264', 96));

const session = streamer.createSession(video, {
  local: { port: 5004 },                    // <-- incoming stream goes here
  remote: { ip: '127.0.0.1', port: 4567 },  // <-- outgoing stream goes here
});

session.getOutgoingStreamTrack().attachTo(session.getIncomingStreamTrack());

FFmpeg streamer:

ffmpeg -stream_loop -1 -re -i video.mp4 -vcodec copy -f rtp "rtp://127.0.0.1:5004"

FFplay receiver:

ffplay -protocol_whitelist file,rtp,udp -i input.sdp

The input.sdp file is needed to indicate port and a codec:

c=IN IP4 127.0.0.1
m=video 4567 RTP 96
a=rtpmap:96 H264/90000

[laricheveg]

@bakoushin thanks, it's pretty clear example. But i would like to achieve another goal.

Let met explain better.

Flow is: webrtc ---> medooze --> ffmpeg --> ffmpeg output (mp4/rtmp/etc)
so using recorder demo, i see loopback, video recorderd in file.

So i suppose we got here webrtc input:

const incomingStream = transport.createIncomingStream(offered);

Here loopback:

const outgoingStream  = transport.createOutgoingStream({
					audio: false,
					video: true
				});

//Get local stream info
const info = outgoingStream.getStreamInfo();

//Copy incoming data from the remote stream to the local one
outgoingStream.attachTo(incomingStream);

//Add local stream info it to the answer
answer.addStream(info);

`
And here:

recorder.record(incomingStream);

i would like to set:

ffmpeg -protocol_whitelist file,rtp,udp -i <<incomingStream>> -vcodec copy -a:c aac -f flv "recorder.mp4"

so based on your example:

const session = streamer.createSession(video, {
  local: { port: 5004 },                    // <-- incomingStream webrtc here
  remote: { ip: '127.0.0.1', port: 4567 },  // <-- outgoing stream goes here
});

or put webrtc like this, idk:

session.getOutgoingStreamTrack().attachTo(incomingStream.getVideoTracks()[0]);

maybe created input.sdp after

and then, probably:

ffmpeg -protocol_whitelist file,rtp,udp -i input.sdp -vcodec copy -a:c aac -f flv "recorder.mp4"

So is it possible (i hope it is) to do something like this? Could you provide please some sample?

Thanks !

[bakoushin]

Just replace FFplay with FFmpeg and you are done:

ffmpeg -protocol_whitelist file,rtp,udp -i input.sdp -c copy -flags +global_header out.mp4

[laricheveg]

Could you please suggest :
1) Video bitrate:
at loopback: targetbitrate 4686399
at ffmpeg output:
frame= 1498 fps= 30 q=-1.0 size= 15616kB time=00:01:02.17 bitrate=2057.4kbits/s speed=1.24x

code for init video session:

function initVideoChannel() {

	const streamer = MediaServer.createStreamer();

	//Create new video session codecs
	const video = new MediaInfo("video","video");

	//Add h264 codec
	video.addCodec(new CodecInfo("h264",96));
	video.setBitrate(50000); //seems like doesn't work
	//Create session for video
	const session = streamer.createSession(video,{
		local  : {
			port: 5004
		},
		remote : {
			ip : "127.0.0.1",
			port: 8321
		}
	});

	return session
}
videoSession.getOutgoingStreamTrack().attachTo(incomingStream.getVideoTracks()[0]);

input.sdp

c=IN IP4 127.0.0.1
m=video 8321 RTP/AVP 96
a=rtpmap:96 H264/90000
a=fmtp:96 packetization-mode=1

How to set bitrate fully passthrough, it speeds up just before 2500kbit ?

2) How to catch audio stream?
I create separate streamer for audio:

function initAudioChannel() {

	const streamer = MediaServer.createStreamer();

	//Create new video session codecs
	const audio = new MediaInfo("audio","audio");

	//Add h264 codec
	audio.addCodec(new CodecInfo("opus",48));

//audio.setDirection(Direction.RECVONLY);						

	//Create session for video
	const session = streamer.createSession(audio,{
		local  : {
			port: 5004
		},
		remote : {
			ip : "127.0.0.1",
			port: 8400
		}
	});

	return session
}
audioSession.getOutgoingStreamTrack().attachTo(incomingStream.getAudioTracks()[0]);

input.sdp:
c=IN IP4 127.0.0.1
m=audio 8400 RTP/AVP 111
a=rtpmap:111 OPUS/48000

But ffmpeg can't catch stream....

const Capabilities = {
	audio : {
		codecs		: ["opus"],
		extensions	: [ "urn:ietf:params:rtp-hdrext:ssrc-audio-level"]
	},
	video : {
		codecs		: ["h264;packetization-mode=1"],
		rtx		: true,
		rtcpfbs		: [
			{ "id": "transport-cc"},
			{ "id": "ccm", "params": ["fir"]},
			{ "id": "nack"},
			{ "id": "nack", "params": ["pli"]}
		],
		extensions	: [ "http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01"]
	}
};

I would like to achieve smth like this:

c=IN IP4 127.0.0.1
m=audio 8400 RTP/AVP 111
a=rtpmap:111 OPUS/48000
m=video 8321 RTP/AVP 96
a=rtpmap:96 H264/90000
a=fmtp:96 packetization-mode=1

and another step:

ffmpeg -protocol_whitelist file,rtp,udp -i input.sdp -c:v copy -acodec aac -flags +global_header out.mp4

Could please suggest @bakoushin ?

[bakoushin]

In this repo they are solving the same problem with another SFU. The author had reported a bug in FFmpeg which prevents storing Opus within MP4.

I neither wasn't able to put Opus into MP4 with FFmpeg.

Maybe using GStreamer could be more fruitful.

[laricheveg]

@bakoushin no matter about Opus. It could be pcmu or smth. Anyway FFmpeg reencode it to aac before write in mp4 wrapper. I even can't
ffmpeg -protocol_whitelist file,rtp,udp -i inputaudio.sdp -c copy -b:a 96k -flags +global_header -loglevel debug out.opus

[laricheveg]

@bakoushin i know that ffmpeg 100% could get pcmu for example, reencode to aac and then store. So my current problem i can't even catch this anyway. Let's forget about mp4 for a while...

So could please provide me with example just how to catch audio and store it. Doesn't matter which webrtc audio was setted ....

[laricheveg]

I have try to use Kurento and with:

'-protocol_whitelist', 'file,udp,rtp,tcp',
 '-i', path.join(__dirname, streamip + '_' + streamport + '.udp'),
'-c:v', 'copy', 
'-acodec', 'aac',
 '-r', '30', 
 '-tune', 'zerolatency',
 '-f', 'flv',
'ouput.mp4'

And it works fine with pcmu or opus forced. But reason i left Kurento that it works unstable with FullHD streaming... So i believe we could apply the same on Medooze

How do you think @bakoushin ?

[laricheveg]

@bakoushin

i add in media soup:

  if (useAudio) {
    cmdCodec += " -map 0:a:0 -c:a aac";
  }

and got fine audio on mp4..... So how do the same in medooze ? 🥇

[murillo128]

Check CodecInfo constructor:
https://medooze.github.io/semantic-sdp-js/#codecinfo

You are setting the payload type to 48:

audio.addCodec(new CodecInfo("opus",48));

but in the sdp you are using 111:

c=IN IP4 127.0.0.1
m=audio 8400 RTP/AVP 111
a=rtpmap:111 OPUS/48000

You can do a wireshark capture to check that the what actual data the streamer session is sending to ffmpeg

[bakoushin]

I've created a quick demo for recording to MP4 trough RTP streams sent to GStreamer/FFmpeg:
https://github.com/bakoushin/medooze-streamer-demo