03.srt

1
00:00:00,000 --> 00:00:04,500
字幕生成：BLACK 字幕校对：凝渊

2
00:00:05,500 --> 00:00:06,700
哈喽大家好

3
00:00:06,700 --> 00:00:08,000
我是ZOMI

4
00:00:08,000 --> 00:00:08,700
那这一节呢

5
00:00:08,700 --> 00:00:13,500
是上一节AI编译器发展阶段的一个延续

6
00:00:13,500 --> 00:00:15,800
来看看一个通用架构

7
00:00:17,000 --> 00:00:19,700
把上一节的三个阶段呢

8
00:00:19,700 --> 00:00:20,700
分成两部分

9
00:00:20,700 --> 00:00:21,300
第一部分呢

10
00:00:21,300 --> 00:00:24,400
先讲AI编译器的一个发展发展的三个阶段

11
00:00:24,400 --> 00:00:24,800
接着呢

12
00:00:24,800 --> 00:00:25,500
今天呢

13
00:00:25,500 --> 00:00:29,200
来讲讲针对第二个阶段里面的一个具体的通用架构

14
00:00:30,000 --> 00:00:31,000
那在开始之前呢

15
00:00:31,000 --> 00:00:32,400
我想重复两个问题

16
00:00:32,400 --> 00:00:36,300
那第一个就是AI框架跟AI编译器到底什么关系

17
00:00:36,300 --> 00:00:38,200
AI框架就是AI编译器吗

18
00:00:38,200 --> 00:00:41,400
还是AI框架和AI编译器有一个划分呢

19
00:00:41,400 --> 00:00:41,800
第二个呢

20
00:00:41,800 --> 00:00:44,700
就是AI领域真的需要编译器吗

21
00:00:44,700 --> 00:00:46,600
像PyTorch的动态图这么灵活

22
00:00:46,600 --> 00:00:47,700
那么多人用

23
00:00:47,700 --> 00:00:49,700
为什么一定要需要编译器呢

24
00:00:50,800 --> 00:00:51,000
好

25
00:00:51,000 --> 00:00:52,000
带着这两个问题

26
00:00:52,000 --> 00:00:54,500
继续来看看AI编译器的一个具体的架构吧

27
00:00:56,700 --> 00:00:58,500
你在教我做事啊

28
00:00:58,500 --> 00:00:59,800
现有AI编译器的架构呢

29
00:00:59,800 --> 00:01:01,600
就是专用编译器的一个架构

30
00:01:01,600 --> 00:01:03,900
刚才谈到了专用编译器的架构呢

31
00:01:03,900 --> 00:01:04,800
在表达上呢

32
00:01:04,800 --> 00:01:06,100
以PyTorch作为标杆

33
00:01:06,100 --> 00:01:08,400
然后对静态图进行转换

34
00:01:08,400 --> 00:01:09,200
性能上呢

35
00:01:09,200 --> 00:01:12,000
希望打开计算图和算子的边界

36
00:01:12,000 --> 00:01:12,700
那架构呢

37
00:01:12,700 --> 00:01:14,900
就由右边所示

38
00:01:14,900 --> 00:01:15,400
前端呢

39
00:01:15,400 --> 00:01:17,200
会对Python的代码进行解析

40
00:01:17,200 --> 00:01:17,600
然后呢

41
00:01:17,600 --> 00:01:18,900
对图层的优化

42
00:01:18,900 --> 00:01:20,200
计算图的优化

43
00:01:20,200 --> 00:01:24,200
然后再到后端生成不同硬件的代码或者执行程序

44
00:01:25,400 --> 00:01:26,000
下面呢

45
00:01:26,000 --> 00:01:29,400
我非常推荐大家去了解一篇文章

46
00:01:29,400 --> 00:01:31,400
叫做Deep Learning Compiler

47
00:01:31,400 --> 00:01:32,700
Comprehensive Survey

48
00:01:32,700 --> 00:01:35,500
就是一个AI编译器的综述

49
00:01:35,500 --> 00:01:36,100
这里面呢

50
00:01:36,100 --> 00:01:37,400
少了一个MindSpore

51
00:01:37,400 --> 00:01:38,300
其实里面呢

52
00:01:38,300 --> 00:01:40,900
主要是分析不同的AI框架

53
00:01:40,900 --> 00:01:41,900
它把AI框架呢

54
00:01:41,900 --> 00:01:44,100
作为的一个前端的表达

55
00:01:44,100 --> 00:01:45,600
但实际上了像TensorFlow

56
00:01:45,600 --> 00:01:48,100
它可能就已经包括了这整一个内容

57
00:01:48,100 --> 00:01:51,600
MindSpore也包括了下面整一套内容

58
00:01:51,600 --> 00:01:52,700
那所以说呢

59
00:01:52,700 --> 00:01:54,800
AI框架跟编译器之间的概念

60
00:01:54,800 --> 00:01:56,400
或者它们之间的一个边界呢

61
00:01:56,400 --> 00:01:58,100
在哪里发出一个思考

62
00:01:59,400 --> 00:02:00,700
那可以看到AI编译器呢

63
00:02:00,700 --> 00:02:02,600
主要是分为前端

64
00:02:02,600 --> 00:02:04,500
还有AI编译器的后端

65
00:02:04,500 --> 00:02:05,900
那这里面的前端呢

66
00:02:05,900 --> 00:02:07,500
包括了一个图层的编译器

67
00:02:07,500 --> 00:02:09,200
还有一个算子层的编译器

68
00:02:09,200 --> 00:02:10,300
那不同层里面呢

69
00:02:10,300 --> 00:02:13,000
有非常多不同的一些IR或者Pass

70
00:02:13,000 --> 00:02:14,200
像DCE呢

71
00:02:14,200 --> 00:02:16,400
就是在之前讲LLVM的时候

72
00:02:16,400 --> 00:02:17,900
死代码消除里面就会有

73
00:02:17,900 --> 00:02:18,600
而这里面呢

74
00:02:18,600 --> 00:02:20,000
也同样引入了非常多

75
00:02:20,000 --> 00:02:23,700
类似于LLVM或者传统编译器的很多的概念

76
00:02:23,700 --> 00:02:24,700
这些所有的内容呢

77
00:02:24,700 --> 00:02:26,700
都会在后面的章节里面

78
00:02:26,700 --> 00:02:28,800
详细的去展开每一个Pattern

79
00:02:28,800 --> 00:02:30,800
或者每一个Pass之间有什么关系

80
00:02:30,800 --> 00:02:31,700
有什么内容

81
00:02:31,700 --> 00:02:34,500
对图产生了哪些影响

82
00:02:34,500 --> 00:02:35,800
下面呢来看看

83
00:02:35,800 --> 00:02:37,100
AI编译器的一个IR

84
00:02:37,100 --> 00:02:38,400
就是中间表达

85
00:02:38,400 --> 00:02:39,300
编译器的IR呢

86
00:02:39,300 --> 00:02:42,000
其实在LLVM那节讲的

87
00:02:42,000 --> 00:02:43,200
应该算比较明白了

88
00:02:43,200 --> 00:02:44,800
但是在AI编译器里面呢

89
00:02:44,800 --> 00:02:46,200
它主要分为两层

90
00:02:46,200 --> 00:02:47,800
第一层呢是High Level的IR

91
00:02:47,800 --> 00:02:49,800
第二层呢是Low Level的IR

92
00:02:49,800 --> 00:02:50,800
High Level的IR呢

93
00:02:50,800 --> 00:02:54,300
主要是特别用来表示计算图

94
00:02:54,300 --> 00:02:56,100
表示神经网络

95
00:02:56,100 --> 00:02:58,200
有了高层对神经网络的表示呢

96
00:02:58,200 --> 00:02:59,900
就可以对神经网络的图

97
00:02:59,900 --> 00:03:01,400
或者神经网络的IR呢

98
00:03:01,400 --> 00:03:03,000
进行一个编译优化

99
00:03:03,000 --> 00:03:04,300
而Low Level的IR呢

100
00:03:04,300 --> 00:03:06,700
更类似于传统的编译器

101
00:03:06,700 --> 00:03:08,200
在更细粒度层面

102
00:03:08,200 --> 00:03:09,700
在代码层面

103
00:03:09,700 --> 00:03:11,500
在指令集层面

104
00:03:11,500 --> 00:03:13,400
进行一些优化

105
00:03:13,400 --> 00:03:14,200
有了IR之后

106
00:03:14,200 --> 00:03:16,900
看一下AI编译器的一个前端

107
00:03:16,900 --> 00:03:18,800
AI框架呢

108
00:03:18,800 --> 00:03:20,000
主要的作用啊

109
00:03:20,000 --> 00:03:21,400
还是一个构图

110
00:03:21,400 --> 00:03:22,600
就是用Python语言

111
00:03:22,600 --> 00:03:24,100
构造了一个计算图

112
00:03:24,100 --> 00:03:26,200
然后丢给AI编译器的前端

113
00:03:26,200 --> 00:03:27,300
AI编译器的前端呢

114
00:03:27,300 --> 00:03:30,800
就对这个计算图进行一些图层的优化

115
00:03:30,800 --> 00:03:31,700
那图层的优化

116
00:03:31,700 --> 00:03:32,900
主要分为三层

117
00:03:32,900 --> 00:03:34,700
第一个是节点级的优化

118
00:03:34,700 --> 00:03:36,100
第二个是块级的优化

119
00:03:36,100 --> 00:03:36,700
第三个呢

120
00:03:36,700 --> 00:03:38,500
是数据级的优化

121
00:03:38,500 --> 00:03:40,400
那逐个来看看节点的优化呢

122
00:03:40,400 --> 00:03:41,300
就会有一些

123
00:03:41,300 --> 00:03:43,200
zero-dim-Tensor elimination

124
00:03:43,200 --> 00:03:45,500
就是零维度的张量的消除

125
00:03:45,500 --> 00:03:46,500
那快级的优化呢

126
00:03:46,500 --> 00:03:48,300
就有代数的简化

127
00:03:48,300 --> 00:03:49,300
常量的折叠

128
00:03:49,300 --> 00:03:50,400
算子的融合

129
00:03:50,400 --> 00:03:53,200
这个是对好几个算子变成一个块

130
00:03:53,200 --> 00:03:55,100
变成一个子图的一个优化

131
00:03:55,100 --> 00:03:55,700
第三个呢

132
00:03:55,700 --> 00:03:57,800
就是数据流级别的优化

133
00:03:57,800 --> 00:03:58,300
ZOMI呢

134
00:03:58,300 --> 00:03:59,800
其实更愿意称它为

135
00:03:59,800 --> 00:04:01,500
Low Level IR的一些优化

136
00:04:01,500 --> 00:04:03,700
例如传统编译器里面的CSE

137
00:04:03,700 --> 00:04:05,300
消除一些公共的子表达式

138
00:04:05,300 --> 00:04:06,500
还有DCE

139
00:04:06,500 --> 00:04:07,500
而AI编译器呢

140
00:04:07,500 --> 00:04:08,700
它也有它自己的后端

141
00:04:08,700 --> 00:04:10,800
那后端进行一些特定的优化

142
00:04:10,800 --> 00:04:12,400
那主要分为三点啊

143
00:04:12,400 --> 00:04:14,200
第一点就是特定的硬件优化

144
00:04:14,200 --> 00:04:16,900
第二点是自动去调整kernel

145
00:04:16,900 --> 00:04:17,500
第三个呢

146
00:04:17,500 --> 00:04:20,500
就是优化整个kernel库

147
00:04:20,500 --> 00:04:20,900
 

148
00:04:20,900 --> 00:04:23,100
现在逐个的来看一下

149
00:04:23,100 --> 00:04:24,500
将会在后面呢

150
00:04:24,500 --> 00:04:26,100
后面的一些内容里面

151
00:04:26,100 --> 00:04:27,600
详细的去展开这里面

152
00:04:27,600 --> 00:04:28,800
所有的每一块内容

153
00:04:28,800 --> 00:04:30,700
这里面我只是给大家一个

154
00:04:30,700 --> 00:04:32,300
统一的review的概念

155
00:04:32,300 --> 00:04:34,100
虽然听不懂这一节完全没有关系

156
00:04:34,100 --> 00:04:36,400
或者搞不清楚这里面的很多名词

157
00:04:36,400 --> 00:04:38,000
也没有关系

158
00:04:38,000 --> 00:04:40,600
瞧你那个作死的样子

159
00:04:40,600 --> 00:04:42,100
现在不是处在一个

160
00:04:42,100 --> 00:04:44,000
硬件架构的黄金十年吗

161
00:04:44,000 --> 00:04:46,700
现在有越来越多的新的加速器的出现

162
00:04:46,700 --> 00:04:49,300
包括DPU NPU各种XPU的出现

163
00:04:49,300 --> 00:04:50,000
不同的硬件

164
00:04:50,000 --> 00:04:52,400
需要对它进行一个特殊的优化

165
00:04:52,400 --> 00:04:53,100
那这个时候呢

166
00:04:53,100 --> 00:04:54,700
可能有两种方式

167
00:04:54,700 --> 00:04:55,200
第一种呢

168
00:04:55,200 --> 00:04:56,500
就是把低级的IR

169
00:04:56,500 --> 00:04:58,600
就是刚才讲的low level的IR

170
00:04:58,600 --> 00:05:00,600
转换成为LLVM IR

171
00:05:00,600 --> 00:05:02,500
利用LLVM的结构

172
00:05:02,500 --> 00:05:03,900
或者LLVM的CodeGen

173
00:05:03,900 --> 00:05:06,300
生成对应的对应硬件的代码

174
00:05:06,300 --> 00:05:07,200
或者执行程序

175
00:05:07,200 --> 00:05:07,800
那第二点呢

176
00:05:07,800 --> 00:05:09,900
就是充分的利用了领域的知识

177
00:05:09,900 --> 00:05:12,600
然后对硬件做一些特殊的优化

178
00:05:12,600 --> 00:05:13,000
这种呢

179
00:05:13,000 --> 00:05:15,000
就是针对特定硬件的优化

180
00:05:15,000 --> 00:05:15,400
第二个呢

181
00:05:15,400 --> 00:05:16,800
就是自动调整

182
00:05:16,800 --> 00:05:17,500
那自动调整

183
00:05:17,500 --> 00:05:19,500
主要是在写kernel的时候

184
00:05:19,500 --> 00:05:20,500
或者写算子

185
00:05:20,500 --> 00:05:21,300
硬件的人呢

186
00:05:21,300 --> 00:05:22,700
不叫他叫算子

187
00:05:22,700 --> 00:05:23,900
因为算子层呢

188
00:05:23,900 --> 00:05:26,300
这个是在框架层的一个概念

189
00:05:26,300 --> 00:05:27,300
但是越往下了

190
00:05:27,300 --> 00:05:28,400
他就不叫算子了

191
00:05:28,400 --> 00:05:29,200
他叫kernel

192
00:05:29,200 --> 00:05:32,000
所以这个概念希望大家能够区分开来

193
00:05:32,000 --> 00:05:32,600
因为呢

194
00:05:32,600 --> 00:05:34,800
现在去写一些kernel的时候

195
00:05:34,800 --> 00:05:38,300
整体的参数的搜索空间是非常大的

196
00:05:38,300 --> 00:05:39,900
怎么对张量进行切分

197
00:05:39,900 --> 00:05:41,500
怎么对数据进行切分

198
00:05:41,500 --> 00:05:43,300
数据应该怎么排布

199
00:05:43,300 --> 00:05:44,100
那这个时候呢

200
00:05:44,100 --> 00:05:46,000
整体的搜索空间非常大

201
00:05:46,000 --> 00:05:46,600
于是呢

202
00:05:46,600 --> 00:05:47,600
现在有两种方法

203
00:05:47,600 --> 00:05:48,100
第一种呢

204
00:05:48,100 --> 00:05:50,400
是利用Halide或者TVM的方式

205
00:05:50,500 --> 00:05:53,600
利用机器学习去做一些搜索优化

206
00:05:53,600 --> 00:05:54,000
第二种呢

207
00:05:54,000 --> 00:05:55,700
就是利用多面体模型

208
00:05:55,700 --> 00:05:57,700
Polyhedral Model进行一个优化的

209
00:05:57,700 --> 00:05:58,500
那这里面呢

210
00:05:58,500 --> 00:05:59,700
说到Polyhedron Model呢

211
00:05:59,700 --> 00:06:02,300
推荐赵洁老师写的一本书

212
00:06:02,300 --> 00:06:05,200
大家可以在京东上面去找到

213
00:06:05,200 --> 00:06:05,600
这个呢

214
00:06:05,600 --> 00:06:08,600
只是给做一个简单的推荐一本好书

215
00:06:08,600 --> 00:06:09,800
而不是卖个广告

216
00:06:09,800 --> 00:06:11,600
希望平台不要把我把我封掉

217
00:06:11,600 --> 00:06:13,900
不然我要给这本书给打打马赛克了

218
00:06:16,100 --> 00:06:16,700
第三点呢

219
00:06:16,700 --> 00:06:18,600
就是优化内核库

220
00:06:18,700 --> 00:06:20,800
公司有个很搞笑的事情

221
00:06:20,800 --> 00:06:22,200
就是把这个内核库啊

222
00:06:22,200 --> 00:06:24,100
叫做内库

223
00:06:24,100 --> 00:06:24,500
这种呢

224
00:06:24,500 --> 00:06:26,800
就是厂商特定的一个内核库

225
00:06:26,800 --> 00:06:27,700
举个简单的例子

226
00:06:27,700 --> 00:06:29,100
就是类似于英伟达呢

227
00:06:29,100 --> 00:06:31,100
就会推出自己的一个

228
00:06:31,100 --> 00:06:33,700
CuDNN, CuBLAS, CuFFT这些库

229
00:06:33,700 --> 00:06:34,300
昇腾呢

230
00:06:34,300 --> 00:06:36,300
就推出了自己的CANN的一个算子库

231
00:06:36,300 --> 00:06:37,600
某些情况下呢

232
00:06:37,600 --> 00:06:39,700
厂商提供的优化内核库呢

233
00:06:39,700 --> 00:06:42,000
可能会比自己AI编译器

234
00:06:42,000 --> 00:06:44,900
去生成的一些算子效率可能会更高

235
00:06:44,900 --> 00:06:46,500
因为经过特殊的优化嘛

236
00:06:46,500 --> 00:06:47,100
最后呢

237
00:06:47,100 --> 00:06:48,400
看一个图啊

238
00:06:48,400 --> 00:06:49,500
这个图是很有意思

239
00:06:49,500 --> 00:06:51,900
也是现在一个最重要的发展阶段

240
00:06:51,900 --> 00:06:54,000
从下往上看

241
00:06:54,000 --> 00:06:54,600
那底下呢

242
00:06:54,600 --> 00:06:56,200
就是不同的硬件

243
00:06:56,200 --> 00:06:58,100
有X86 ARM, Viska, GPU

244
00:06:58,100 --> 00:06:59,900
还有华为的达芬奇架构

245
00:06:59,900 --> 00:07:00,800
华为的昇腾

246
00:07:00,800 --> 00:07:01,400
这里面呢

247
00:07:01,400 --> 00:07:03,200
就有非常多的硬件

248
00:07:03,200 --> 00:07:05,200
端边云的硬件都有

249
00:07:05,200 --> 00:07:06,400
那有了这些硬件呢

250
00:07:06,400 --> 00:07:09,200
需要把算法给编译到硬件上面

251
00:07:09,200 --> 00:07:10,400
真正的去执行

252
00:07:10,400 --> 00:07:10,800
于是呢

253
00:07:10,800 --> 00:07:11,800
就会有一个kernel层

254
00:07:11,800 --> 00:07:12,300
kernel层呢

255
00:07:12,300 --> 00:07:16,000
就是提供一些真正的执行的kernel的算子

256
00:07:16,000 --> 00:07:16,600
那这里面呢

257
00:07:16,600 --> 00:07:19,800
就推出了TC, TVM, AutoKernel, CANN, cuDNN

258
00:07:19,800 --> 00:07:21,900
推出了一些不同级别的kernel的优化库

259
00:07:21,900 --> 00:07:22,900
或者编译器

260
00:07:22,900 --> 00:07:24,300
不管是AA编译器也好

261
00:07:24,300 --> 00:07:25,200
手工的也好

262
00:07:25,200 --> 00:07:27,400
都是对硬件进行赋能的

263
00:07:27,400 --> 00:07:28,400
再往上层走呢

264
00:07:28,400 --> 00:07:31,100
就是Graph Level的一个IR或者编译器

265
00:07:31,100 --> 00:07:31,800
那这里面呢

266
00:07:31,800 --> 00:07:33,600
就有一个Facebook的长颈鹿啦

267
00:07:33,600 --> 00:07:34,100
N-Graph

268
00:07:34,100 --> 00:07:37,000
还有谷歌的XLA, TVM, 还有MindSpore

269
00:07:37,000 --> 00:07:38,200
再往上层走呢

270
00:07:38,200 --> 00:07:40,900
就是深度学习的框架

271
00:07:40,900 --> 00:07:42,000
就是AI框架

272
00:07:42,000 --> 00:07:44,000
那可能会有这早期的Caffe

273
00:07:44,000 --> 00:07:44,600
TensorFlow

274
00:07:44,600 --> 00:07:45,400
还有PyTorch

275
00:07:45,500 --> 00:07:47,500
MindSpore, JX, OneFlow

276
00:07:47,500 --> 00:07:49,700
当然了还有国内的好几个厂商

277
00:07:49,700 --> 00:07:50,900
Jittor, Paddle-Paddle

278
00:07:50,900 --> 00:07:52,800
那这些呢就是AI框架

279
00:07:52,800 --> 00:07:55,600
现在整个AI编译器的全栈呢

280
00:07:55,600 --> 00:07:58,100
主要是涉及到这些内容

281
00:07:58,900 --> 00:08:01,600
最后呢又到了大家喜闻乐见的环节

282
00:08:01,600 --> 00:08:03,000
来做一个总结

283
00:08:03,000 --> 00:08:05,600
今天呢主要给大家去分享和汇报了

284
00:08:05,600 --> 00:08:08,000
现在AI编译器呢主要分为三个阶段

285
00:08:08,000 --> 00:08:09,800
从stage1, stage2专用的

286
00:08:09,800 --> 00:08:12,300
然后再到未来可能会走向的通用

287
00:08:12,300 --> 00:08:14,300
第二个呢就是简单的去展开了

288
00:08:14,300 --> 00:08:16,000
AI编译器的一个通用的架构

289
00:08:16,000 --> 00:08:18,500
包括里面的IR是怎么是表达的

290
00:08:18,500 --> 00:08:19,300
分为两层

291
00:08:19,300 --> 00:08:20,100
第一层high level

292
00:08:20,100 --> 00:08:21,300
第二层是low level

293
00:08:21,300 --> 00:08:23,000
然后呢又分为编译器的前端

294
00:08:23,000 --> 00:08:24,200
和编译器的后端

295
00:08:24,200 --> 00:08:25,300
那编译器的前端呢

296
00:08:25,300 --> 00:08:26,800
可能做了好几件事情

297
00:08:26,800 --> 00:08:27,800
那编译器的后端呢

298
00:08:27,800 --> 00:08:29,400
可能会更聚焦于

299
00:08:29,400 --> 00:08:31,300
Kernel的生成自动化的一些工作

300
00:08:31,300 --> 00:08:32,200
好了谢谢各位

301
00:08:32,200 --> 00:08:33,300
拜了个拜

302
00:08:33,300 --> 00:08:35,000
卷的不行了卷的不行了

303
00:08:35,000 --> 00:08:36,800
记得一键三连加关注哦

304
00:08:36,800 --> 00:08:38,400
所有的内容都会开源在

305
00:08:38,400 --> 00:08:40,300
下面这条链接里面

306
00:08:40,300 --> 00:08:41,600
摆了个掰