Skip to content

Commit

Permalink
docs: fix incorrect variable names in SSM notes
Browse files Browse the repository at this point in the history
  • Loading branch information
danbev committed Aug 11, 2024
1 parent ce9725b commit e17820d
Showing 1 changed file with 20 additions and 21 deletions.
41 changes: 20 additions & 21 deletions notes/state-space-models.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,14 +27,13 @@ input matrix `B`, the output matrix `C`, and the feedthrough matrix `D`.

The formula for S4 models is:
```
xₜ₊₁ = Axₜ + Buₜ
hₜ₊₁ = Axₜ + Bxₜ
yₜ = Cxₜ + Duₜ
Xₜ₊₁ = is the state at time t+1
hₜ₊₁ = is the state at time t+1
A = state transition matrix
Xₜ = is the state at time t
B = input matrix
Uₜ = input vector
xₜ = is the input at time t
B = input projection matrix
C = output matrix
D = feedthrough matrix
```
Expand All @@ -56,46 +55,46 @@ B = 1
C = 2
D = 1
x₀ = 0 (hidden state similar to short-term memory IN RNN)
h₀ = 0 (hidden state similar to short-term memory IN RNN)
u = [2, 3, 1]
We have the forumlas
xₜ₊₁ = Axₜ + Buₜ
yₜ = Cxₜ + Duₜ
hₜ₊₁ = Ahₜ + Bxₜ
yₜ = Chₜ + Dxₜ
And we plug in our values:
A x₀ B u
x₁ = (0.5 * 0) + (1 * 2) = 2
x₁ = 2 ---+
A h₀ B x
h₁ = (0.5 * 0) + (1 * 2) = 2
h₁ = 2 ---+
C x₁ D u
C h₁ D x
y₁ = (2 * 2) + (1 * 2) = 4
y₁ = 4
That was one forward pass. The hidden state (x₁) has been updated to 2 and
That was one forward pass. The hidden state (h₁) has been updated to 2 and
the output (y₁) is 4.
And now we do it again with the new values, notice that x₁ is now 2 instead of
And now we do it again with the new values, notice that h₁ is now 2 instead of
0:
A x₁ B u
x₂ = (0.5 * 2) + (1 * 3) = 4
x₂ = 4 ---+
A h₁ B x
h₂ = (0.5 * 2) + (1 * 3) = 4
h₂ = 4 ---+
C x₂ D u
C h₂ D x
y₂ = (2 * 4) + (1 * 3) = 11
y₂ = 11
A x₂ B u
A h₂ B x
x₃ = (0.5 * 4) + (1 * 1) = 3
x₃ = 3 ---+
C x₃ D u
C h₃ D x
y₃ = (2 * 3) + (1 * 1) = 7
y₃ = 7
```
Each input token updates the state of the system based on the previous state
and the current input.

x is similar to the hidden state in RNNs and allows information to be passed
h is similar to the hidden state in RNNs and allows information to be passed
from one step to the next. y is the output of the system at each time step.

__wip__
Expand Down

0 comments on commit e17820d

Please sign in to comment.