04.srt

1
00:00:00,000 --> 00:00:05,400
字幕生成：qiaokai 字幕校对：mkwei

2
00:00:06,333 --> 00:00:06,933
哈喽

3
00:00:06,933 --> 00:00:08,666
大家好我是ZOMI

4
00:00:08,666 --> 00:00:09,866
今天来到

5
00:00:09,866 --> 00:00:11,966
AI编译器系列里面的前端优化

6
00:00:11,966 --> 00:00:14,466
来讲讲数据布局转换

7
00:00:15,866 --> 00:00:17,133
在之前的内容里面呢

8
00:00:17,133 --> 00:00:18,966
讲了一个图层的IR

9
00:00:18,966 --> 00:00:19,999
就是计算图

10
00:00:20,000 --> 00:00:22,933
怎么转换成AI编译器的前端的输入

11
00:00:22,966 --> 00:00:24,699
接着去了解了一下

12
00:00:24,700 --> 00:00:26,866
算子融合的一个基础的原理

13
00:00:26,866 --> 00:00:28,266
和算子融合的方式

14
00:00:28,300 --> 00:00:31,400
今天呢来讲讲数据布局的转换

15
00:00:31,400 --> 00:00:33,800
其实很多人对数据布局其实

16
00:00:33,800 --> 00:00:35,166
不是说非常的敏感

17
00:00:35,166 --> 00:00:36,299
但是数据布局呢

18
00:00:36,300 --> 00:00:38,100
对AI编译器来说是

19
00:00:38,100 --> 00:00:40,000
非常重要的一个环节

20
00:00:40,133 --> 00:00:43,699
数据布局主要是layout transformation

21
00:00:43,766 --> 00:00:44,299
然后呢

22
00:00:44,300 --> 00:00:46,666
今天会分开几个内容去介绍的

23
00:00:47,000 --> 00:00:48,666
如果十来分钟介绍不完

24
00:00:48,666 --> 00:00:50,499
会拆开两个内容

25
00:00:50,500 --> 00:00:51,400
那首先看看

26
00:00:51,400 --> 00:00:53,400
今天要给大家汇报

27
00:00:53,400 --> 00:00:54,333
哪些知识点

28
00:00:54,333 --> 00:00:57,133
首先第一个就是数据内存的排布

29
00:00:57,133 --> 00:00:57,599
第二个呢

30
00:00:57,600 --> 00:00:59,566
去看看张量的一个数据

31
00:00:59,566 --> 00:01:01,333
具体内存是怎么排布的

32
00:01:01,500 --> 00:01:01,966
再往下呢

33
00:01:01,966 --> 00:01:03,766
会去看看NCHW

34
00:01:03,800 --> 00:01:06,166
和NHWC之间的数据排布

35
00:01:06,166 --> 00:01:07,499
具体有什么作用

36
00:01:07,500 --> 00:01:08,533
有什么不同

37
00:01:08,533 --> 00:01:09,566
对于硬件来说

38
00:01:09,566 --> 00:01:11,999
更青睐于采用哪种方式

39
00:01:12,366 --> 00:01:14,999
往下就是自己华为昇腾的产品

40
00:01:15,000 --> 00:01:17,466
去看看华为昇腾的一个数据

41
00:01:17,466 --> 00:01:18,499
布局的排布

42
00:01:18,500 --> 00:01:19,733
到底是怎么样的

43
00:01:19,800 --> 00:01:23,966
又怎么如何跟NCHW或者NHWC进行融合

44
00:01:24,100 --> 00:01:24,666
最后呢

45
00:01:24,666 --> 00:01:27,933
去看看AI编译器的数据布局转换

46
00:01:27,933 --> 00:01:29,366
是怎么做优化

47
00:01:30,333 --> 00:01:30,766
现在呢

48
00:01:30,766 --> 00:01:33,933
回头来看看在哪个位置

49
00:01:34,000 --> 00:01:37,133
前端把Python的代码呢进行解析

50
00:01:37,133 --> 00:01:38,466
那这个解析的过程呢

51
00:01:38,466 --> 00:01:41,333
是通过AI框架去实现的

52
00:01:41,366 --> 00:01:43,533
解析完之后得到一个计算图

53
00:01:43,533 --> 00:01:46,333
那计算图呢就传给AI编译器的

54
00:01:46,533 --> 00:01:48,866
第一层图层的IR的优化

55
00:01:48,866 --> 00:01:52,066
现在在这个地方数据布局转换

56
00:01:52,066 --> 00:01:53,499
作为图层IR

57
00:01:53,500 --> 00:01:55,333
前端优化的一个pass

58
00:01:56,900 --> 00:01:57,400
接下来呢

59
00:01:57,400 --> 00:01:59,000
看一下第一个内容

60
00:01:59,000 --> 00:02:01,200
数据内存的排布

61
00:02:02,400 --> 00:02:03,600
ZOMI老师你好

62
00:02:03,600 --> 00:02:06,166
我想问一下什么是内存对齐啊

63
00:02:06,166 --> 00:02:09,133
为什么要对内存进行对齐呢

64
00:02:10,000 --> 00:02:12,266
哎这位同学问的问题非常好啊

65
00:02:12,266 --> 00:02:14,199
这也是第一个内容

66
00:02:14,366 --> 00:02:15,066
哈哈哈

67
00:02:15,066 --> 00:02:17,166
还是我自己问自己答的

68
00:02:17,166 --> 00:02:19,766
第一个内容呢就是讲数据内存的排布

69
00:02:19,933 --> 00:02:21,333
首先要了解一个概念

70
00:02:21,333 --> 00:02:22,799
就是内存的对齐

71
00:02:22,800 --> 00:02:26,000
和数据在内存里面的位置是相关的

72
00:02:26,000 --> 00:02:28,533
那现在看看左边的这个图

73
00:02:28,900 --> 00:02:30,600
这里面呢是memory

74
00:02:30,600 --> 00:02:32,266
就是内存的地址

75
00:02:32,266 --> 00:02:33,666
这些存的都是

76
00:02:33,666 --> 00:02:35,199
内存的一个具体的地址

77
00:02:35,300 --> 00:02:38,166
而右边的这个呢就是实际的数据

78
00:02:38,166 --> 00:02:40,666
数据呢是存放在内存

79
00:02:40,700 --> 00:02:42,166
某个地址里面的

80
00:02:42,333 --> 00:02:45,399
而内存呢是以字节为单位进行存储

81
00:02:45,400 --> 00:02:46,566
如果一个变量

82
00:02:46,566 --> 00:02:48,566
或者一个数据的内存的地址呢

83
00:02:48,566 --> 00:02:50,533
刚好等于它的长度的倍数

84
00:02:50,533 --> 00:02:52,999
那这个呢就称为自然对齐

85
00:02:53,133 --> 00:02:53,866
理论上呢

86
00:02:53,866 --> 00:02:56,533
其实可以从任意地址开始进行

87
00:02:56,533 --> 00:02:57,999
读取和写入的

88
00:02:58,300 --> 00:03:00,066
如果数据内存没有对齐呢

89
00:03:00,066 --> 00:03:01,266
可能就比较麻烦

90
00:03:01,266 --> 00:03:02,599
读取数据的时候呢

91
00:03:02,600 --> 00:03:03,600
就没有那么方便

92
00:03:03,600 --> 00:03:05,766
要对数据呢进行一个偏移再读取

93
00:03:05,766 --> 00:03:06,599
然后再合并

94
00:03:07,200 --> 00:03:10,066
尽管现代的内存呢是以字节为单位的

95
00:03:10,066 --> 00:03:12,066
但是呢现在的处理器呢

96
00:03:12,066 --> 00:03:14,799
是以字为单位进行一个读写和访问的

97
00:03:14,800 --> 00:03:16,200
CPU啊

98
00:03:16,200 --> 00:03:18,400
或者不管是Inter还是arm的CPU

99
00:03:18,500 --> 00:03:20,933
还是按字节块的方式来去读取的

100
00:03:20,933 --> 00:03:23,099
而一般有几种读取方式

101
00:03:23,100 --> 00:03:24,733
就是以字节2

102
00:03:24,933 --> 00:03:28,666
4 8 16这种字节的方式为粒度呢

103
00:03:28,666 --> 00:03:30,133
进行内存的读写

104
00:03:30,133 --> 00:03:31,899
既然CPU和系统呢

105
00:03:31,900 --> 00:03:34,466
是根据这些字节为单位进行读取

106
00:03:34,700 --> 00:03:37,000
所以呢尽可能的把数据

107
00:03:37,066 --> 00:03:39,299
根据这个单位进行对齐

108
00:03:39,300 --> 00:03:41,666
才能够高效的去利用硬件

109
00:03:41,666 --> 00:03:44,166
充分的发挥硬件的性能

110
00:03:44,200 --> 00:03:45,366
现在呢来看看

111
00:03:45,366 --> 00:03:48,499
假设数据内存没有对齐

112
00:03:48,500 --> 00:03:50,666
会产生一个什么样的情况

113
00:03:50,766 --> 00:03:52,066
那下面这个图呢

114
00:03:52,133 --> 00:03:55,099
以4字节存储的粒度为单位

115
00:03:55,100 --> 00:03:56,200
读取一个int

116
00:03:56,200 --> 00:03:57,100
一个变量

117
00:03:57,333 --> 00:04:00,766
int呢有4个字节32比特

118
00:04:00,766 --> 00:04:01,599
处理器去读取的时候呢

119
00:04:01,600 --> 00:04:04,366
会默认从4的倍数的地址开始进行

120
00:04:04,366 --> 00:04:08,599
读取假设我这个位置的地址呢是0X0000

121
00:04:08,900 --> 00:04:12,166
这个位置的地址呢可能是0X0004

122
00:04:12,333 --> 00:04:14,499
假设现在的数据存储呢

123
00:04:14,500 --> 00:04:16,133
是没有以四字节为单位

124
00:04:16,133 --> 00:04:17,199
进行一个存储的

125
00:04:17,266 --> 00:04:18,766
现在去读取的时候呢

126
00:04:18,766 --> 00:04:22,366
先读取上面红色的这块的前三个Byte

127
00:04:22,500 --> 00:04:23,200
然后呢

128
00:04:23,200 --> 00:04:25,400
再读取这蓝色的这一块的

129
00:04:25,400 --> 00:04:26,533
第一个Low Byte

130
00:04:26,533 --> 00:04:29,133
就是第一个最低的位置地址

131
00:04:29,133 --> 00:04:31,399
就是我红色的这个位置

132
00:04:31,533 --> 00:04:34,499
这个时候呢我就需要访问两次内存的

133
00:04:34,500 --> 00:04:37,333
第一次呢就是读取红色的高位的地址

134
00:04:37,333 --> 00:04:40,166
第二次呢就是读取蓝色的低位的地址

135
00:04:40,400 --> 00:04:41,600
读取完之后呢

136
00:04:41,600 --> 00:04:43,366
第一次从0地址读取

137
00:04:43,366 --> 00:04:45,866
然后把首个字节呢剔除出来

138
00:04:45,866 --> 00:04:49,099
第二次呢从4地址开始读取只读取

139
00:04:49,533 --> 00:04:50,799
首个字节

140
00:04:50,800 --> 00:04:53,733
最后呢把这两块地址呢合并起来

141
00:04:53,800 --> 00:04:54,800
变成一个

142
00:04:54,800 --> 00:04:56,600
int 32位就取一个数

143
00:04:56,733 --> 00:04:58,666
假设现在只取了一个1,000的数

144
00:04:58,666 --> 00:05:00,933
但是呢读内存要读两次

145
00:05:00,933 --> 00:05:03,499
这是非常耗费硬件的资源的

146
00:05:03,800 --> 00:05:05,166
刚才1,000这个数呢

147
00:05:05,166 --> 00:05:07,966
已经存在红色这块地址

148
00:05:07,966 --> 00:05:08,866
这种情况呢

149
00:05:08,866 --> 00:05:10,733
叫做做了一个数据

150
00:05:10,733 --> 00:05:12,966
跟内存一个严格对齐

151
00:05:13,333 --> 00:05:16,699
读取的时候呢起始位置作为4的倍数

152
00:05:16,700 --> 00:05:18,600
然后只需要进行一次读取

153
00:05:18,600 --> 00:05:20,000
就可以从memory

154
00:05:20,000 --> 00:05:21,933
然后把数据搬到一个寄存器

155
00:05:21,933 --> 00:05:22,533
里面了

156
00:05:22,533 --> 00:05:24,366
最后呢给SLA进行计算

157
00:05:25,566 --> 00:05:27,699
以字节的大小为粒度呢进行内存访问

158
00:05:27,700 --> 00:05:28,733
其实有两个好处

159
00:05:28,733 --> 00:05:30,733
第一个呢就是提升访问速率

160
00:05:30,733 --> 00:05:32,066
为什么提升访问速率

161
00:05:32,066 --> 00:05:33,699
刚才其实已经用一个

162
00:05:33,700 --> 00:05:34,933
简单的例子讲了

163
00:05:34,933 --> 00:05:35,333
第二个呢

164
00:05:35,333 --> 00:05:37,766
就是保存数据的一个原子性

165
00:05:37,933 --> 00:05:40,599
现在简单的来看看两个的好处

166
00:05:40,600 --> 00:05:41,900
现在CPU呢

167
00:05:41,900 --> 00:05:43,966
大部分都有多个高级的缓存呢

168
00:05:43,966 --> 00:05:44,399
数据呢

169
00:05:44,400 --> 00:05:46,866
必须通过这些高级缓存去读取数据的

170
00:05:46,866 --> 00:05:49,366
而以字节大小粒度呢进行内存访问呢

171
00:05:49,366 --> 00:05:52,166
可以整体提升CPU的吞吐量

172
00:05:52,166 --> 00:05:54,933
那第二点就是原子性的问题

173
00:05:55,066 --> 00:05:55,466
CPU呢

174
00:05:55,466 --> 00:05:57,666
可以在一个对齐的内存上面进行操作

175
00:05:57,666 --> 00:05:58,733
这意味着没有

176
00:05:58,733 --> 00:06:00,933
指令可以中断访问操作

177
00:06:00,933 --> 00:06:02,366
对于很多没有锁的数据

178
00:06:02,366 --> 00:06:03,699
或者数据结构来说呢

179
00:06:03,700 --> 00:06:05,533
可以提升并发的正确性

180
00:06:05,533 --> 00:06:07,066
那这个就是两个好处

181
00:06:08,066 --> 00:06:10,166
接下来呢回到AI的概念

182
00:06:10,166 --> 00:06:10,999
就是张量

183
00:06:11,000 --> 00:06:12,766
张量的数据呢又是如何布局

184
00:06:12,766 --> 00:06:13,933
刚才只是讲了

185
00:06:13,933 --> 00:06:15,333
在普通的X86上面

186
00:06:15,333 --> 00:06:17,299
数据是怎么进行一个存储

187
00:06:17,300 --> 00:06:19,366
的数据的存储方式呢

188
00:06:19,366 --> 00:06:20,499
是根据数据在

189
00:06:20,500 --> 00:06:22,100
主存里面的存储的方式

190
00:06:22,100 --> 00:06:23,500
之上作为基础的

191
00:06:25,733 --> 00:06:28,133
现在来看看张量和内存的

192
00:06:28,133 --> 00:06:29,099
排布的方式

193
00:06:29,100 --> 00:06:31,100
那张量呢是一个多维的数组

194
00:06:31,100 --> 00:06:33,366
先看看简单的一个数呢

195
00:06:33,366 --> 00:06:34,499
叫做标量

196
00:06:34,500 --> 00:06:35,400
他只有一个数

197
00:06:35,400 --> 00:06:36,900
而有多个数的时候呢

198
00:06:36,900 --> 00:06:38,000
叫做向量

199
00:06:38,000 --> 00:06:39,500
就是一维的张量

200
00:06:39,600 --> 00:06:40,566
如果数据呢

201
00:06:40,566 --> 00:06:41,666
是一个矩阵的时候呢

202
00:06:41,666 --> 00:06:43,133
叫做二维的张量

203
00:06:43,133 --> 00:06:44,333
当数据

204
00:06:44,333 --> 00:06:46,299
假设像这样一张图片

205
00:06:46,300 --> 00:06:49,866
那图片呢有长宽高还有一个通道数RGB

206
00:06:49,900 --> 00:06:52,333
那这个时候呢就会变成三维的张量

207
00:06:52,500 --> 00:06:54,600
当然了在神经网络处理的时候呢

208
00:06:54,666 --> 00:06:56,133
会有一个Batch Size

209
00:06:56,133 --> 00:06:57,999
作为批数据处理的方式

210
00:06:58,000 --> 00:07:00,266
那这个时候呢就变成四维的张量了

211
00:07:00,466 --> 00:07:01,533
而作为0维张量

212
00:07:01,533 --> 00:07:02,799
单单一个数呢

213
00:07:02,800 --> 00:07:04,600
以字节为单位进行存储

214
00:07:04,700 --> 00:07:06,300
向量里面每一个元素呢

215
00:07:06,300 --> 00:07:08,200
都是按字节为单位进行存储

216
00:07:08,366 --> 00:07:10,266
但是当维度越来越高

217
00:07:10,266 --> 00:07:11,866
内存数据地址呢

218
00:07:11,866 --> 00:07:13,166
是按列进行排布的

219
00:07:13,166 --> 00:07:14,566
所以存储1维张量的时候

220
00:07:14,566 --> 00:07:15,999
大家比较好理解

221
00:07:16,000 --> 00:07:17,766
这里面 5呢我存在一个地址

222
00:07:17,766 --> 00:07:18,699
4存在一个地址

223
00:07:18,700 --> 00:07:20,766
3存在一个地址2存在一个地址

224
00:07:20,766 --> 00:07:22,299
但是呢当矩阵的时候

225
00:07:22,300 --> 00:07:23,166
可能

226
00:07:23,466 --> 00:07:26,133
1存在一个地址里面不断的往下排

227
00:07:26,133 --> 00:07:28,599
2存在一个地址里面不断的往下排

228
00:07:28,600 --> 00:07:30,100
3也是但是

229
00:07:30,100 --> 00:07:32,500
张量的维度越来越高的时候

230
00:07:32,500 --> 00:07:33,966
是先存列了

231
00:07:34,133 --> 00:07:35,333
还是先存行呢

232
00:07:35,366 --> 00:07:38,333
还是先存channel的数量呢

233
00:07:38,333 --> 00:07:40,266
如果再加一个维度的时候

234
00:07:40,266 --> 00:07:41,733
数据应该怎么存呢

235
00:07:41,733 --> 00:07:42,199
这个时候

236
00:07:42,200 --> 00:07:43,333
就引起张量

237
00:07:43,333 --> 00:07:45,066
里内存排布的一个问题啦

238
00:07:46,933 --> 00:07:47,599
现在呢

239
00:07:47,600 --> 00:07:49,066
先以一个简单形状的

240
00:07:49,066 --> 00:07:51,133
3*2*2的一个3维张量

241
00:07:51,133 --> 00:07:53,399
进行一个存储排布的一个讲解

242
00:07:53,400 --> 00:07:54,600
或者作为一个例子

243
00:07:54,766 --> 00:07:57,933
假设先按行优先进行数据排布

244
00:07:57,966 --> 00:08:00,666
红色的这个呢叫做按行进行排布

245
00:08:00,666 --> 00:08:03,366
现在是一个3*2*2的3维张量

246
00:08:03,366 --> 00:08:05,099
那会以行进行排布

247
00:08:05,100 --> 00:08:06,900
先把红色的排起来

248
00:08:06,900 --> 00:08:08,900
然后再把橙色的排起来

249
00:08:08,900 --> 00:08:11,466
接着呢再排绿色的这一个位置

250
00:08:11,466 --> 00:08:14,499
最后呢再排底下蓝色的这个位置

251
00:08:14,500 --> 00:08:15,766
而现在看到

252
00:08:15,766 --> 00:08:18,266
虽然作为3*2*2的3维张量

253
00:08:18,266 --> 00:08:19,866
实际呢在内存里面呢

254
00:08:19,866 --> 00:08:20,933
就是排成一排

255
00:08:20,933 --> 00:08:23,166
因为内存是以列进行排布的

256
00:08:23,166 --> 00:08:26,333
当然最终呢他可能是一排过就打束的

257
00:08:26,333 --> 00:08:28,766
但是没关系这不影响理解

258
00:08:29,566 --> 00:08:30,399
接下来再看

259
00:08:30,400 --> 00:08:32,366
以列优先进行一个数据排布

260
00:08:32,366 --> 00:08:35,066
列优先就是先按列的进行走

261
00:08:35,066 --> 00:08:37,133
那先存红的然后再存橙的

262
00:08:37,166 --> 00:08:38,599
先存红的再存橙的

263
00:08:38,600 --> 00:08:42,733
就变成红橙红橙红橙这种排布方式

264
00:08:42,733 --> 00:08:44,733
接下来呢对第二个channel

265
00:08:44,733 --> 00:08:46,699
就是后面的那个维度进行排列

266
00:08:46,700 --> 00:08:48,200
先存绿的再存蓝的

267
00:08:48,200 --> 00:08:49,500
先存绿的再存蓝的

268
00:08:49,500 --> 00:08:52,566
就变成绿蓝绿蓝绿蓝

269
00:08:52,566 --> 00:08:54,366
这种存储排布方式

270
00:08:54,400 --> 00:08:56,400
所以看到的张量呢

271
00:08:56,400 --> 00:08:58,133
它的形状有非常多

272
00:08:58,133 --> 00:09:00,766
但是呢按行排布按列排布

273
00:09:01,000 --> 00:09:01,366
后面呢

274
00:09:01,366 --> 00:09:04,166
就会有非常多的内存排布的方式

275
00:09:05,100 --> 00:09:07,933
现在呢以2*2*2的一个三维张量

276
00:09:07,933 --> 00:09:09,166
看看数据

277
00:09:09,166 --> 00:09:11,133
内存排布有非常多的方式

278
00:09:11,133 --> 00:09:13,299
那假设现在有三个方向

279
00:09:13,300 --> 00:09:15,400
第一个方向呢是横向的

280
00:09:15,533 --> 00:09:16,366
这个方向呢

281
00:09:16,366 --> 00:09:18,999
叫做D1 第二个呢是以列方向

282
00:09:19,000 --> 00:09:21,100
这个方向呢叫做D2

283
00:09:21,100 --> 00:09:22,000
那第三个呢

284
00:09:22,000 --> 00:09:24,966
以channel这个轴为方向

285
00:09:24,966 --> 00:09:26,566
叫做D3

286
00:09:26,566 --> 00:09:29,166
以D1 D2 D3这种方式

287
00:09:29,166 --> 00:09:31,699
就先排D1再排D2再排D3

288
00:09:31,733 --> 00:09:33,399
那这种方式进行排布呢

289
00:09:33,400 --> 00:09:35,000
有一种这种方式

290
00:09:35,000 --> 00:09:36,866
假设以D1 D3 D2

291
00:09:36,866 --> 00:09:38,399
这种排布呢

292
00:09:38,400 --> 00:09:39,566
有这种方式

293
00:09:39,566 --> 00:09:42,133
先排了红的然后再排一个绿的

294
00:09:42,133 --> 00:09:44,533
然后再排橙色的最后排蓝色的

295
00:09:44,533 --> 00:09:49,566
所以可以看到这里面有123456

296
00:09:49,600 --> 00:09:51,266
六种数据的排布

297
00:09:51,266 --> 00:09:52,933
仅仅是2*2的三维张量

298
00:09:52,933 --> 00:09:56,599
就已经有六种数据排布的方式了

299
00:09:57,366 --> 00:09:58,466
我这一辈子

300
00:09:58,766 --> 00:10:01,366
再也不会爱上任何一个女人了

301
00:10:03,066 --> 00:10:05,766
了解完张量的数据排布之后呢

302
00:10:05,766 --> 00:10:08,199
迎来了一个比较重要的内容

303
00:10:08,200 --> 00:10:11,600
就是NCHW和NHWC

304
00:10:11,866 --> 00:10:13,666
到底是采用NCHW

305
00:10:13,700 --> 00:10:16,333
还是NHWC这种数据格式呢

306
00:10:16,366 --> 00:10:18,366
往下去看一看

307
00:10:18,400 --> 00:10:19,000
首先呢

308
00:10:19,000 --> 00:10:21,766
还是回顾刚才的一个概念

309
00:10:21,766 --> 00:10:23,266
就是尽管数据啊

310
00:10:23,266 --> 00:10:24,999
实际上都是一样的数据

311
00:10:25,200 --> 00:10:26,600
但是不同的顺序呢

312
00:10:26,600 --> 00:10:29,200
会导致数据的访问的性能

313
00:10:29,200 --> 00:10:30,166
是不一样的

314
00:10:30,166 --> 00:10:31,099
所以这里面呢

315
00:10:31,100 --> 00:10:34,066
就会衍生了很多不同数据的排布方式

316
00:10:34,133 --> 00:10:37,299
那NCHW还是NHWC呢

317
00:10:37,333 --> 00:10:38,466
在这里面呢

318
00:10:38,466 --> 00:10:40,166
简单的去看看

319
00:10:40,166 --> 00:10:43,099
每一个维度代表的是什么意思

320
00:10:43,333 --> 00:10:45,899
N呢就是BatchSize的一个数量

321
00:10:45,900 --> 00:10:47,366
批处理的数量

322
00:10:47,466 --> 00:10:49,699
H呢就是图片的高度

323
00:10:49,700 --> 00:10:51,700
W图片的宽度

324
00:10:51,700 --> 00:10:54,700
而channel呢就是图片的通道数

325
00:10:54,733 --> 00:10:56,999
也可以称为特征图的通道数

326
00:10:57,000 --> 00:10:59,166
这里面呢只是以CNN卷积神经网络

327
00:10:59,166 --> 00:11:01,099
就是专门处理图像

328
00:11:01,100 --> 00:11:02,600
进行介绍的

329
00:11:02,800 --> 00:11:05,300
现在来看看第一个数据存储的方式

330
00:11:05,300 --> 00:11:07,400
NCHW

331
00:11:07,533 --> 00:11:08,366
这种格式

332
00:11:08,666 --> 00:11:10,099
以图片作为例子

333
00:11:10,100 --> 00:11:12,200
假设呢这是一个单通道的图片

334
00:11:12,200 --> 00:11:14,133
后面呢也是一个单通道的图片

335
00:11:14,133 --> 00:11:16,266
在后面也是一个单通道的图片

336
00:11:16,266 --> 00:11:18,499
这个时候呢我同一通道的数值

337
00:11:18,533 --> 00:11:19,966
呢是连续的排布的

338
00:11:19,966 --> 00:11:21,399
也就是123456

339
00:11:21,566 --> 00:11:23,799
然后同时排在第一个位置

340
00:11:23,800 --> 00:11:26,300
接着第二个通道的数据呢连续排布

341
00:11:26,300 --> 00:11:28,933
最后第三个通道的数据再连续排布

342
00:11:28,966 --> 00:11:30,199
这种排布方式呢

343
00:11:30,200 --> 00:11:30,400
更

344
00:11:30,400 --> 00:11:32,966
适合需要对每个通道单独运算的操作

345
00:11:32,966 --> 00:11:34,133
例如MaxPooling

346
00:11:34,366 --> 00:11:35,099
计算的时候呢

347
00:11:35,100 --> 00:11:36,900
ALU就是计算单元

348
00:11:36,900 --> 00:11:39,100
单独的对单个通道进行计算

349
00:11:39,100 --> 00:11:41,266
接着呢再对下一个通道进行计算

350
00:11:41,266 --> 00:11:43,133
这种存储方式的缺点就是

351
00:11:43,133 --> 00:11:44,566
需要的内存比较大

352
00:11:44,566 --> 00:11:46,799
可能需要把整个通道的数量呢

353
00:11:46,800 --> 00:11:49,000
都加载到内存或者显存里面

354
00:11:49,166 --> 00:11:51,399
所以说它比较适合GPU进行运算

355
00:11:51,400 --> 00:11:53,533
利用GPU的内存和带宽比较大

356
00:11:53,533 --> 00:11:55,533
并行能力特别强的这种特性

357
00:11:55,566 --> 00:11:56,966
这么说还是有点抽象

358
00:11:56,966 --> 00:11:58,933
接下来再往下看一看

359
00:11:58,933 --> 00:12:00,733
假设现在把刚才的

360
00:12:00,766 --> 00:12:03,333
数据呢按NCHW的方式进行处理

361
00:12:03,500 --> 00:12:05,666
现在呢数据已经攒成一排了

362
00:12:05,666 --> 00:12:06,799
接着我在计算的时候

363
00:12:06,800 --> 00:12:08,200
我先算第一个通道

364
00:12:08,200 --> 00:12:09,666
然后乘以0.299

365
00:12:09,700 --> 00:12:12,200
然后第二个通道我再进行一个操作

366
00:12:12,500 --> 00:12:13,100
第三次呢

367
00:12:13,100 --> 00:12:16,100
我整一个通道再进行进行一个操作

368
00:12:16,133 --> 00:12:16,766
这个时候呢

369
00:12:16,766 --> 00:12:18,933
就每一个通道充分的利用了

370
00:12:18,933 --> 00:12:20,066
GPU的并行能力

371
00:12:20,066 --> 00:12:21,899
最后把三个通道计算的结果呢

372
00:12:21,900 --> 00:12:22,533
进行相加

373
00:12:22,533 --> 00:12:24,266
得到整体的灰度值

374
00:12:24,300 --> 00:12:26,466
这种方式呢就是GPU最擅长的一个操作

375
00:12:26,466 --> 00:12:28,999
利用GPU的并行能力

376
00:12:29,000 --> 00:12:30,000
进行一个计算

377
00:12:31,366 --> 00:12:32,933
现在看看第二种方式

378
00:12:32,933 --> 00:12:35,099
就是NHWC的排布方式

379
00:12:35,133 --> 00:12:37,266
NHWC的排布方式很有意思

380
00:12:37,266 --> 00:12:40,133
就是每一个通道单独存储一个数据

381
00:12:40,133 --> 00:12:42,666
每一个通道单独再从存储一个数据

382
00:12:42,666 --> 00:12:48,199
我从173173到284284这种方式去进行存储的

383
00:12:48,300 --> 00:12:49,466
这种存储方式呢

384
00:12:49,466 --> 00:12:51,766
更适合那些不需要对通道数进行

385
00:12:51,766 --> 00:12:53,166
逐一操作的方式

386
00:12:53,200 --> 00:12:54,766
例如1*1的卷积

387
00:12:54,966 --> 00:12:57,866
而且比较适合多核CPU进行计算的

388
00:12:57,866 --> 00:13:00,499
CPU呢会对每一个元素进行计算

389
00:13:00,766 --> 00:13:02,799
假设1是CPUa进行计算

390
00:13:02,800 --> 00:13:05,500
7呢是CPU2进行计算

391
00:13:05,500 --> 00:13:07,933
13呢是CPU3进行计算

392
00:13:08,266 --> 00:13:10,899
每一个元素呢交给一个独立的CPU

393
00:13:10,900 --> 00:13:12,166
独立的核进行计算

394
00:13:12,200 --> 00:13:13,800
提高CPU的并发率

395
00:13:13,800 --> 00:13:16,600
而大家都知道CPU是分为多级缓存的

396
00:13:16,600 --> 00:13:19,000
存储寄存器带宽相对较小

397
00:13:19,000 --> 00:13:21,100
但是每个数据的计算时间比较低

398
00:13:21,100 --> 00:13:22,333
临时空间也很小

399
00:13:22,500 --> 00:13:24,666
采用NHWC这种存储方式呢

400
00:13:24,666 --> 00:13:26,733
有便于CPU呢采取异步的方式

401
00:13:26,733 --> 00:13:29,133
边读边计算来减少访存的时间

402
00:13:29,133 --> 00:13:30,766
控制起来也比较灵活

403
00:13:31,666 --> 00:13:32,099
那下面呢

404
00:13:32,100 --> 00:13:34,733
来看看一个更加具体的一个例子

405
00:13:34,733 --> 00:13:35,966
或者更形象的一个例子

406
00:13:35,966 --> 00:13:37,599
假设刚才把NHWC

407
00:13:37,600 --> 00:13:39,100
的数据已经排起来了

408
00:13:39,100 --> 00:13:41,466
接下来现在有3个CPU的核

409
00:13:41,466 --> 00:13:42,366
3个CPU的核

410
00:13:42,366 --> 00:13:44,866
分别单独的去处理其中一个数值

411
00:13:44,866 --> 00:13:46,466
然后对他进行一个操作

412
00:13:46,466 --> 00:13:48,599
操作完之后呢再进行一个累加

413
00:13:48,600 --> 00:13:50,966
最后得到图像的灰度值

414
00:13:51,266 --> 00:13:55,066
这种就是NHWC数据排布的一个方式

415
00:13:56,766 --> 00:13:58,166
由于数据呢

416
00:13:58,166 --> 00:14:00,866
在内存里面呢只能是通过线性的处理

417
00:14:00,866 --> 00:14:03,166
这四个维度的数据的存储的方式呢

418
00:14:03,166 --> 00:14:04,699
是有对应的顺序的

419
00:14:04,700 --> 00:14:05,766
不同的AI框架呢

420
00:14:05,766 --> 00:14:07,499
会使用不同的存储方式呢

421
00:14:07,500 --> 00:14:09,766
对数据或者对特征图

422
00:14:09,933 --> 00:14:11,066
feature map进行存储

423
00:14:11,066 --> 00:14:12,699
以NPU或者GPU为例子呢

424
00:14:12,700 --> 00:14:13,700
就是PyTroch

425
00:14:13,700 --> 00:14:14,500
和MindSpore呢

426
00:14:14,533 --> 00:14:17,399
默认使用NCHW这种格式方式存储的

427
00:14:17,400 --> 00:14:20,566
NHWC就是先存Batch然后再存channel

428
00:14:20,666 --> 00:14:22,333
再从宽高

429
00:14:22,400 --> 00:14:24,466
那就是第一种方式

430
00:14:24,700 --> 00:14:25,600
第二种方式呢

431
00:14:25,600 --> 00:14:27,966
NHWC呢就是TensorFlow

432
00:14:28,000 --> 00:14:29,600
一开始时候采用的一种方式

433
00:14:29,600 --> 00:14:30,500
就是先存Batch

434
00:14:30,533 --> 00:14:32,566
然后Height Width然后再存Channel

435
00:14:32,566 --> 00:14:35,199
就是下面的这种存储方式啦

436
00:14:38,133 --> 00:14:40,566
接下来添加一个额外的知识点

437
00:14:40,566 --> 00:14:42,966
就是连续和非连续的问题

438
00:14:42,966 --> 00:14:45,466
让看看什么叫做连续的张量存储

439
00:14:45,466 --> 00:14:47,666
123456 那这种方式呢

440
00:14:47,666 --> 00:14:49,699
就是连续的数据处理方式

441
00:14:49,933 --> 00:14:52,699
第二种就是非连续的张量存储方式

442
00:14:52,700 --> 00:14:55,266
就是假设现在有一个2*3的矩阵

443
00:14:55,266 --> 00:14:57,966
那数据存储的时候呢是142536

444
00:14:57,966 --> 00:14:59,999
就是以这种方式进行存储的

445
00:15:00,000 --> 00:15:02,333
那这种呢叫做非连续的张量

446
00:15:02,933 --> 00:15:04,533
对于一些非连续的张量了

447
00:15:04,533 --> 00:15:06,133
执行操作变换的时候呢

448
00:15:06,133 --> 00:15:08,799
需要重新的去开辟内存空间

449
00:15:08,800 --> 00:15:11,200
也就开辟一个一样大小的内存空间

450
00:15:11,200 --> 00:15:12,333
然后进行处理的

451
00:15:12,533 --> 00:15:13,933
这种非连续的存储方式呢

452
00:15:13,933 --> 00:15:15,266
其实是不太建议的

453
00:15:15,266 --> 00:15:17,266
更建议是按照连续的存储方式

454
00:15:17,266 --> 00:15:17,999
但有时候呢

455
00:15:18,000 --> 00:15:18,866
在处理的时候

456
00:15:18,866 --> 00:15:19,999
不可避免的会出现

457
00:15:20,000 --> 00:15:21,500
非连续的张量的方式

458
00:15:21,500 --> 00:15:22,200
所以大家知道

459
00:15:22,200 --> 00:15:23,366
内存空间

460
00:15:23,366 --> 00:15:24,866
对实际的AI框架来说

461
00:15:24,866 --> 00:15:25,866
是非常有用的

462
00:15:25,866 --> 00:15:27,666
可能大家在平时写代码

463
00:15:27,666 --> 00:15:28,966
或者写算法的时候呢

464
00:15:29,200 --> 00:15:31,066
没有太注意数据的存储方式

465
00:15:31,066 --> 00:15:32,866
但是作为AI框架的开发者

466
00:15:32,866 --> 00:15:34,766
或者对系统的工程师来说呢

467
00:15:34,800 --> 00:15:37,600
这是一个非常严峻非常大的一个挑战

468
00:15:38,933 --> 00:15:40,366
好了那回顾一下

469
00:15:40,366 --> 00:15:43,066
今天呢讲了一个数据内存的排布

470
00:15:43,066 --> 00:15:45,266
一般是以字节为单位进行一个存储的

471
00:15:45,266 --> 00:15:46,566
这是因为硬件

472
00:15:46,566 --> 00:15:47,966
逻辑地址所决定的

473
00:15:48,200 --> 00:15:48,966
那第二个点呢

474
00:15:48,966 --> 00:15:51,666
讲了张量的一个数据的排布方式

475
00:15:51,666 --> 00:15:54,266
张量的数据排布方式其实有非常多的

476
00:15:54,266 --> 00:15:55,766
刚才以一个2*2*2

477
00:15:55,766 --> 00:15:57,266
三维的张量进行一个例子呢

478
00:15:57,266 --> 00:15:59,299
就已经有六种数据的排布方式了

479
00:15:59,500 --> 00:16:01,166
每种数据排布方式对

480
00:16:01,166 --> 00:16:02,299
硬件的计算

481
00:16:02,300 --> 00:16:03,600
其实是有影响的

482
00:16:03,600 --> 00:16:05,266
于是呢以NCHW

483
00:16:05,266 --> 00:16:08,733
和NHWC这两种不同的硬件存储方式

484
00:16:08,733 --> 00:16:09,933
进行了一个介绍

485
00:16:10,166 --> 00:16:11,466
像NHWC呢

486
00:16:11,466 --> 00:16:12,766
更适合在NPU

487
00:16:12,766 --> 00:16:14,799
或者GPU这些加速芯片进行计算的

488
00:16:14,800 --> 00:16:16,933
那NHWC这种方式呢

489
00:16:16,933 --> 00:16:18,266
更适合在CPU

490
00:16:18,266 --> 00:16:19,666
上面进行一个存储的

491
00:16:20,000 --> 00:16:21,933
今天的内容呢到此为止

492
00:16:21,933 --> 00:16:23,166
好了谢谢各位

493
00:16:23,166 --> 00:16:24,166
拜了个拜

494
00:16:24,966 --> 00:16:25,866
卷的不行了

495
00:16:25,866 --> 00:16:26,733
卷的不行了

496
00:16:26,733 --> 00:16:27,866
记得一键三连加关注