Skip to content

Commit 03a5d2e

Browse files
committed
Update bit format section
1 parent 2fbc46a commit 03a5d2e

File tree

1 file changed

+16
-5
lines changed

1 file changed

+16
-5
lines changed

page.md

Lines changed: 16 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -76,14 +76,25 @@ When we talk about `fp8` models, we typically only are talking about the **weigh
7676

7777
### fp8 bit format
7878

79-
There are a number of different `fp8` formats; the most common is `float8_e4m3fn`. Here are some facts about it:
79+
There are a number of different `fp8` formats; the most common is `float8_e4m3fn`. Here are the bit patterns for the f8 and f16 formats:
80+
81+
| Format | Bit Pattern | `INF` Support |
82+
| ------------- | --------------- | ----------- |
83+
| float8_e4m3fn | ⚫🟩🟩🟩🟩🟥🟥🟥 ||
84+
| float8_e5m2fn | ⚫🟩🟩🟩🟩🟩🟥🟥 ||
85+
| bfloat16 | ⚫🟩🟩🟩🟩🟩🟩🟩🟥🟥🟥🟥🟥🟥🟥 ||
86+
| float16 | ⚫🟩🟩🟩🟩🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥 ||
87+
88+
where: ⚫ = Sign bit, 🟩 = Exponent bit, 🟥 = Mantissa (fraction) bit
89+
90+
Here are some facts about `float8_e4m3fn`:
8091

8192
1. This format has `1` sign bit, `4` bits for exponent (`e4`), and `3` bits for mantissa (`m3`)
8293
2. Values can be between `[-448, +448]`
83-
1. 256 representable values
84-
3. Cannot represent `infinity` (the `fn` postfix stands for "finite numbers only" - there are other fp8 formats that do support infinity)
85-
4. Can represent `NaN`
86-
5. Model parameters are typically stored using this format (note that `inf` is not usually present in pretrained model parameters)
94+
3. There are `256` representable values
95+
4. `infinity` **not** supported (the `fn` postfix stands for "finite numbers only" - there are other fp8 formats that do support infinity)
96+
5. `NaN` supported
97+
6. Model parameters are typically stored using this format (note that `inf` is not usually present in pretrained model parameters)
8798

8899
<details>
89100
<summary>

0 commit comments

Comments
 (0)