-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
8 changed files
with
146 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
# 神经网络 | ||
> 一种强大的计算模型 | ||
也称人工神经网络ANN(Artificial Neural Network),是20世纪80年代以来人工智能领域的一个研究热点。 | ||
它是对人脑神经元网络进行抽象化处理,建立的一种简单模型,并按不同的连接方式组成不同的网络。 | ||
|
||
神经网络由大量的节点(神经元)相互连接构成,每个节点代表一种特定的输出函数,称为激励函数(activation function)。激励函数的作用使得神经元具有非线性性,以更好的模拟实际问题的复杂性。 | ||
每两个节点之间的连接都代表一个对于通过该连接信号的加权值,称之为权重,相当于人工神经网络的记忆。 | ||
|
||
网络的输出则依赖连接方式、权重值和激励函数的不同而不同。工作就是输入数据,处理数据,给出一个结果作为输出。 | ||
|
||
神经网络一般分成三层, | ||
- 输入层,负责接收外部数据 | ||
- 隐含层,对数据进行处理 | ||
- 输出层,负责最终结果输出 | ||
|
||
神经网络的实现有多种 | ||
|
||
## 前馈神经网络 | ||
|
||
当前的输入只依赖于前一层节点的输出,与之前的网络输出状态无关 | ||
|
||
## 反馈神经网络 | ||
|
||
将当前输出再次接到输入层,使得输入层不仅仅取上一层节点的输出 | ||
|
||
## RNN(Recurrent Neural Network) | ||
|
||
循环神经网络,是一种递归式的神经网络结构,通过在序列的每个位置上共享权重,将当前位置的输入与前一时刻的隐藏状态进行串联处理,是一种用于处理序列数据的神经网络。 | ||
RNN能记住之前的信息,并将其应用于当前的计算中,这使得处理自然语言、语音识别、时间序列预测等数据有较好的优势。 | ||
|
||
RNN的缺陷有梯度消失和梯度爆炸,会导致处理长序列数据时性能下降,为解决这个问题提出了改进方法 | ||
- 长短时记忆网络LSTM | ||
- 门控循环单牙GRU | ||
|
||
## Transformer | ||
是一种基于自注意力的序列建模模型,它的每层都由多头注意力机制和前馈神经网络组成,注意力机制来建模序列中的依赖关系,每个输入元素都与序列中的其他元素进行交互,并根据交互结果来调整自身的表示。 | ||
|
||
Transformer因并行和长期依赖建模优势,在自然语言领域表现出色, | ||
|
||
- [神经网络算法 —— 一文搞懂Transformer ](https://developer.aliyun.com/article/1462200?utm_content=g_1000391552&accounttraceid=5b0de7cbee3d4e04b732246795313dcegdxe) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,48 @@ | ||
# GPU | ||
> Graphics Processing Unit | ||
- Asynchronous accelerator for graphics异步图形加速 | ||
- parallel problem并行处理 | ||
1. if a single ALU take a branch, all ALUs will take it.一个分支所有都执行这个分支 | ||
- GPU通过驱动获取API,意味着不能直接获取硬件信息 | ||
- cache | ||
1. constant cache | ||
2. R/W cache | ||
3. Texture cache | ||
4. ROP(Render OutPut) cache | ||
|
||
渲染流程上分immediate mode和tiled-based GPU,各自有各种的优劣势 | ||
```js | ||
// immediate mode | ||
foreach(triangle) | ||
foreach(fragment in triangle) | ||
load FBO data(color, depth, ...) | ||
call fragment shader | ||
store new FBO data | ||
// tiled-based | ||
foreach(fragment) | ||
load FBO data(color, depth, ...) | ||
foreach(triangle) | ||
call fragment shader | ||
store new FBO data | ||
// tile based rendering | ||
foreach(tile) | ||
load tile FBO data(color, depth, ...) | ||
foreach(triangle in tile) | ||
foreach(fragment in triangle in tile) | ||
call fragment shader | ||
store new tile FBO data | ||
``` | ||
## 概念 | ||
|
||
|
||
## where gpu sits in the system | ||
|
||
理想化其概念, | ||
|
||
|
||
## 参考 | ||
|
||
- [GPU Architectures A CPU Perspective](https://courses.cs.washington.edu/courses/cse471/13sp/lectures/GPUsStudents.pdf) | ||
- [CPU vs. GPU Key Differences & Uses Explained](https://www.run.ai/guides/multi-gpu/cpu-vs-gpu) | ||
- [关于他的一篇GPU Architecture文章](https://github.com/Kangz) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
<!DOCTYPE html> | ||
<html> | ||
<head> | ||
<title>Depth to View</title> | ||
<script src="./html.config.js" defer></script> | ||
<script src="./third/pinyin-pro.js" defer></script> | ||
</script> | ||
</head> | ||
<body> | ||
<div class="container-xl"> | ||
<h1>拼音翻译</h1> | ||
<div> | ||
<h4 id="section1">生日</h4> | ||
<div> | ||
<textarea id="idSrc" rows="3"></textarea> | ||
<textarea id="idDst" rows="3" readonly></textarea> | ||
</div> | ||
</div> | ||
<div></div> | ||
<script type="module"> | ||
const { pinyin } = pinyinPro; | ||
const elTextareaSrc = document.getElementById('idSrc'); | ||
const elTextareaDst = document.getElementById('idDst'); | ||
elTextareaSrc.addEventListener('input', (event)=>{ | ||
elTextareaDst.textContent = pinyin(elTextareaSrc.textContent); | ||
}) | ||
</script> | ||
</div> | ||
</body> | ||
</html> |
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters