how to concatenate the feature maps of 3D convolution and the features maps of 2D convolution

[qianngli]

Dear authors,
I have read your code, but I have one question about the concatenting connection about 3/2D convolution.

The paper (MiCT: Mixed 3D/2D Convolutional Tube for Human Action Recognition) try to select one channel ( a tensor with NCWH) to ahieve 2D convolution operation and select several channels (a tensor with NCDW*H) to achieve 3D convolution operation. I don't konw how to concatenate the feature maps of 3D convolution and the features maps of 2D convolution. If convenient, please answer this question.

Regards,
Lee

[gongsuming]

你好啊，我想问一下，代码里CSV文件您有吗？有的话发我一份吧
谢谢

[qianngli]

sorry, I don't have it gongsuming <[email protected]> 于2020年1月3日周五 下午4:52写道：

…

你好啊，我想问一下，代码里CSV文件您有吗？有的话发我一份吧 谢谢 — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#3?email_source=notifications&email_token=ANJ3LIZJ3Z5YJCABVBFITKTQ334F3A5CNFSM4JZKZQW2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEIAT5VI#issuecomment-570506965>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ANJ3LI5K5E3D6VTJA5OITADQ334F3ANCNFSM4JZKZQWQ> .

[gongsuming]

那再打扰一下，您跑起这个程序了吗？ 或者说您自己写了data-loader程序？
谢谢

[fmahoudeau]

Hello,
let’s take for example the initial 3D & 2D convolutions.

• Assuming a batch size of 128 and input sequences of 16 frames, the input has shape 128x3x16x160x160.
• The first Conv3D has stride 1 along the temporal dimension and 2 along the spatial dimensions. It has 64 kernels so it outputs feature maps of shape 128x64x16x80x80.
• The first Conv2D also has stride 2 and 64 kernels but requires 4D input tensors. The input is reshaped to 2048x3x160x160 by stacking the frames of the 128 video sequences along the batch dimension (using function _to_4d_tensor). The output has shape 2048x64x80x80. The videos sequences are unstacked to obtain shape 128x64x16x80x80 (using function _to_5d_tensor)
• The 2 tensors can now be sumed.

The MiCT blocks work the same way. I hope it clarifies.

[gongsuming]

Thank you very much!