关于rtp&h264解析问题 #98

simman · 2023-10-25T01:43:18Z

simman
Oct 25, 2023

感谢大佬提供的库，不知道是否能提供一下帮助。

1、使用了 example/example_rtsp_client_rtp_over_udp.go ；
2、测试过其他 rtsp_server（如: mediamtx），写入文件后可以正常使用vlc播放；
3、当用指定的rtsp_server拉流，然后写文件后，发现vlc无法解析，使用 ffmpeg -i video.h264 -loglevel trace 后, 输出信息如下：

ffmpeg version 6.0 Copyright (c) 2000-2023 the FFmpeg developers
  built with Apple clang version 14.0.3 (clang-1403.0.22.14.1)
  configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/6.0_1 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libaribb24 --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libsvtav1 --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox --enable-audiotoolbox --enable-neon
  libavutil      58.  2.100 / 58.  2.100
  libavcodec     60.  3.100 / 60.  3.100
  libavformat    60.  3.100 / 60.  3.100
  libavdevice    60.  1.100 / 60.  1.100
  libavfilter     9.  3.100 /  9.  3.100
  libswscale      7.  1.100 /  7.  1.100
  libswresample   4. 10.100 /  4. 10.100
  libpostproc    57.  1.100 / 57.  1.100
Splitting the commandline.
Reading option '-i' ... matched as input url with argument 'video.h264'.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument 'trace'.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option loglevel (set logging level) with argument trace.
Successfully parsed a group of options.
Parsing a group of options: input url video.h264.
Successfully parsed a group of options.
Opening an input file: video.h264.
[NULL @ 0x131104080] Opening 'video.h264' for reading
[file @ 0x6000011d4000] Setting default whitelist 'file,crypto,data'
[h264 @ 0x131104080] Format h264 detected only with low score of 1, misdetection possible!
[h264 @ 0x131104080] Before avformat_find_stream_info() pos: 0 bytes read:1048576 seeks:0 nb_streams:1
[h264 @ 0x131004790] non-existing PPS 0 referenced
[extract_extradata @ 0x6000019d0460] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 1
[h264 @ 0x131004790] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 1
[h264 @ 0x131004790] non-existing PPS 0 referenced
[h264 @ 0x131004790] decode_slice_header error
[h264 @ 0x131004790] no frame!
[h264 @ 0x131004790] non-existing PPS 0 referenced
[extract_extradata @ 0x6000019d0460] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 1
[h264 @ 0x131004790] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 1
[h264 @ 0x131004790] non-existing PPS 0 referenced
[h264 @ 0x131004790] decode_slice_header error
[h264 @ 0x131004790] no frame!
[h264 @ 0x131004790] non-existing PPS 0 referenced
[extract_extradata @ 0x6000019d0460] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 1
[h264 @ 0x131004790] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 1
[h264 @ 0x131004790] non-existing PPS 0 referenced
[h264 @ 0x131004790] decode_slice_header error
[h264 @ 0x131004790] no frame!
[h264 @ 0x131004790] non-existing PPS 0 referenced
[extract_extradata @ 0x6000019d0460] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 1
[h264 @ 0x131004790] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 1
[h264 @ 0x131004790] non-existing PPS 0 referenced
[h264 @ 0x131004790] decode_slice_header error
[h264 @ 0x131004790] no frame!
[h264 @ 0x131004790] non-existing PPS 0 referenced
[extract_extradata @ 0x6000019d0460] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 1
[h264 @ 0x131004790] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 1
[h264 @ 0x131004790] non-existing PPS 0 referenced
[h264 @ 0x131004790] decode_slice_header error
[h264 @ 0x131004790] no frame!
[h264 @ 0x131004790] non-existing PPS 0 referenced
[extract_extradata @ 0x6000019d0460] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 1
[h264 @ 0x131004790] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 1
[h264 @ 0x131004790] non-existing PPS 0 referenced
[h264 @ 0x131004790] decode_slice_header error
[h264 @ 0x131004790] no frame!
[h264 @ 0x131004790] non-existing PPS 0 referenced
[extract_extradata @ 0x6000019d0460] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 1
[h264 @ 0x131004790] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 1
[h264 @ 0x131004790] non-existing PPS 0 referenced
[h264 @ 0x131004790] decode_slice_header error
[h264 @ 0x131004790] no frame!
[h264 @ 0x131004790] non-existing PPS 0 referenced
[extract_extradata @ 0x6000019d0460] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 1
[h264 @ 0x131004790] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 1
[h264 @ 0x131004790] non-existing PPS 0 referenced
[h264 @ 0x131004790] decode_slice_header error
[h264 @ 0x131004790] no frame!
[h264 @ 0x131104080] stream 0: start_time: NOPTS duration: NOPTS
[h264 @ 0x131104080] format: start_time: NOPTS duration: NOPTS (estimate from bit rate) bitrate=0 kb/s
[h264 @ 0x131104080] Could not find codec parameters for stream 0 (Video: h264, 1 reference frame, none): unspecified size
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
[h264 @ 0x131104080] After avformat_find_stream_info() pos: 1296434 bytes read:1296434 seeks:0 frames:206
Input #0, h264, from 'video.h264':
  Duration: N/A, bitrate: N/A
  Stream #0:0, 206, 1/1200000: Video: h264, 1 reference frame, none, 25.08 fps, 25 tbr, 1200k tbn
Successfully opened the file.
At least one output file must be specified
[AVIOContext @ 0x131204080] Statistics: 1296434 bytes read, 0 seeks

4、使用ffplay、vlc、openRTSP 可以正常播放第三步的rtsp流；
5、下面的rtp_data.zip是我在 cli.sesss[track.TrackName].rtpSess.Read(buf) 后，直接写的rtp文件。
6、因为一些限制，rtsp_server端是无法进行改动的，只能拉流端做适配。

rtp_data.zip

Answered by yapingcat

Oct 27, 2023

有条件的话，去除 "00 00 01" 或 "00 00 00 01"吧，现在是不符合rfc标准的

View full answer

yapingcat · 2023-10-25T07:33:49Z

yapingcat
Oct 25, 2023
Maintainer

看ffmpeg的输出是没有pps，有抓包吗，可以直接抓个包

0 replies

yapingcat · 2023-10-25T07:46:04Z

yapingcat
Oct 25, 2023
Maintainer

rtp打包h264的时候，看样子没有把 00 00 01 去掉，这样gomedia识别出来的Nalu type 只有0，如果rtp流中不存在sps pps，gomedia只有在I帧的时候才会把sps pps (sdp中解析出来的sps pps) 带上，如果估计是这个原因

0 replies

yapingcat · 2023-10-25T08:08:29Z

yapingcat
Oct 25, 2023
Maintainer

在HandleDescribe 函数里面加入一下代码，解析sdp里面的sps pps,然后把sps pps 写入到文件开头在播放试试

	for _, media := range sdpcontext.Medias {
		tmp := strings.ToLower(media.EncodeName)
		switch tmp {
		case "h264":
			fmtpHandle := sdp.NewH264FmtpParam()
			fmtpHandle.Load(media.Attrs["fmtp"])
			sps, pps := fmtpHandle.GetSpsPps()
		case "h265":
			fmtpHandle := sdp.NewH265FmtpParam()
			fmtpHandle.Load(media.Attrs["fmtp"])
			vps, sps, pps := fmtpHandle.GetVpsSpsPps()
		}

	}

1 reply

simman Oct 25, 2023
Author

好像还是不行，https://github.com/simman/share 这里面有wireshark抓包的数据；

yapingcat · 2023-10-25T09:08:54Z

yapingcat
Oct 25, 2023
Maintainer

这个err 有打印吗，应该会刷 unsupport h264 rtp packet type

err = track.Input(buf[:r], false)
if err != nil {
    fmt.Println(err)
    break
}

1 reply

simman Oct 25, 2023
Author

我在UnPack里

case packType == 0:
		return nil

yapingcat · 2023-10-25T09:16:16Z

yapingcat
Oct 25, 2023
Maintainer

这种情况，rtp没法解析出sps pps

gomedia/go-rtsp/rtp/rtp-h264.go

Line 251 in 67da098

case 0 < packType && packType < 24:

在本地把这行改成 case packType < 24: 然后再试一下

1 reply

simman Oct 25, 2023
Author

加上 https://github.com/yapingcat/gomedia/discussions/98#discussioncomment-7378410，然后改了 case packType < 24: 保存文件后可以正常播了。我再多试几次，先感谢了！～～

yapingcat · 2023-10-26T01:05:09Z

yapingcat
Oct 26, 2023
Maintainer

#98 (comment)

这个可以不用加

1 reply

simman Oct 26, 2023
Author

👌，还有一个和本库无关的问题。

我想把rtsp转成webrtc, 目前使用的是 github.com/pion/webrtc/v4 这个包，我在 https://github.com/yapingcat/gomedia/blob/main/example/example_rtsp_client_rtp_over_udp.go#L132-L133C4 直接 videoTrack.write 写入，pion 那边应该也会存在packType的问题吧，目前是无法正常播放的，所以是不是丢给pion的时候，可以处理下这个udp包？

videoTrack 是通过pion的 videoTrack, err := webrtc.NewTrackLocalStaticRTP(webrtc.RTPCodecCapability{MimeType: webrtc.MimeTypeH264}, "video", "pion") 创建的。

yapingcat · 2023-10-26T04:17:18Z

yapingcat
Oct 26, 2023
Maintainer

👌，还有一个和本库无关的问题。

我想把rtsp转成webrtc, 目前使用的是 github.com/pion/webrtc/v4 这个包，我在 https://github.com/yapingcat/gomedia/blob/main/example/example_rtsp_client_rtp_over_udp.go#L132-L133C4 直接 videoTrack.write 写入，pion 那边应该也会存在packType的问题吧，目前是无法正常播放的，所以是不是丢给pion的时候，可以处理下这个udp包？

videoTrack 是通过pion的 videoTrack, err := webrtc.NewTrackLocalStaticRTP(webrtc.RTPCodecCapability{MimeType: webrtc.MimeTypeH264}, "video", "pion") 创建的。

用 TrackLocalStaticSample 把，直接写入h264 frame

5 replies

simman Oct 26, 2023
Author

videoTrack, _ := webrtc.NewTrackLocalStaticSample()
t.OnSample(func(sample rtsp.RtspSample) {
  mediaSample := media.Sample{
	  Data: sample.Sample,
	  //Timestamp: time.UnixMilli(int64(sample.Timestamp)),
	  Duration: time.Second,
	  //PacketTimestamp:    nil,
	  //PrevDroppedPackets: 0,
	  //Metadata:           nil,
  }
  _ := videoTrack.WriteSample(mediaSample)
}

直接这样写么？我用了个标准的rtsp，写frame 的方式无法播放；

yapingcat Oct 26, 2023
Maintainer

duration 写的不太对，写个40毫秒试试

simman Oct 27, 2023
Author

可以播放，但是画面会跳来跳去的，如果是server端做适配，是不是需要打包时去除 "00 00 01" 或 "00 00 00 01" 的开始码，把其他数据封包为 RTP 包即可以兼容 gomedia 了？

yapingcat Oct 27, 2023
Maintainer

有条件的话，去除 "00 00 01" 或 "00 00 00 01"吧，现在是不符合rfc标准的

Answer selected by simman

simman Oct 31, 2023
Author

感谢，已完成！～

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

关于rtp&h264解析问题 #98

{{title}}

Replies: 7 comments 9 replies

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

关于rtp&h264解析问题 #98

simman Oct 25, 2023

Replies: 7 comments · 9 replies

yapingcat Oct 25, 2023 Maintainer

yapingcat Oct 25, 2023 Maintainer

yapingcat Oct 25, 2023 Maintainer

simman Oct 25, 2023 Author

yapingcat Oct 25, 2023 Maintainer

simman Oct 25, 2023 Author

yapingcat Oct 25, 2023 Maintainer

simman Oct 25, 2023 Author

yapingcat Oct 26, 2023 Maintainer

simman Oct 26, 2023 Author

yapingcat Oct 26, 2023 Maintainer

simman Oct 26, 2023 Author

yapingcat Oct 26, 2023 Maintainer

simman Oct 27, 2023 Author

yapingcat Oct 27, 2023 Maintainer

simman Oct 31, 2023 Author

simman
Oct 25, 2023

Replies: 7 comments 9 replies

yapingcat
Oct 25, 2023
Maintainer

yapingcat
Oct 25, 2023
Maintainer

yapingcat
Oct 25, 2023
Maintainer

simman Oct 25, 2023
Author

yapingcat
Oct 25, 2023
Maintainer

simman Oct 25, 2023
Author

yapingcat
Oct 25, 2023
Maintainer

simman Oct 25, 2023
Author

yapingcat
Oct 26, 2023
Maintainer

simman Oct 26, 2023
Author

yapingcat
Oct 26, 2023
Maintainer

simman Oct 26, 2023
Author

yapingcat Oct 26, 2023
Maintainer

simman Oct 27, 2023
Author

yapingcat Oct 27, 2023
Maintainer

simman Oct 31, 2023
Author