Skip to content

Commit

Permalink
fix typo, intermediate_source/dynamic_quantization_bert_tutorial.rst (#โ€ฆ
Browse files Browse the repository at this point in the history
  • Loading branch information
hadh93 authored Sep 1, 2024
1 parent 80824ea commit 0a68544
Showing 1 changed file with 4 additions and 4 deletions.
8 changes: 4 additions & 4 deletions intermediate_source/dynamic_quantization_bert_tutorial.rst
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,7 @@ BERT ์˜ ์‚ฌ์ƒ์€ ์–ธ์–ด ํ‘œํ˜„์„ ์‚ฌ์ „ํ•™์Šตํ•˜๊ณ , ๋ฌธ์ œ์— ํŠนํ™”๋œ ๋งค
๊ฐ€๋Šฅํ•œ ์ ๊ฒŒ ์‚ฌ์šฉํ•˜๋ฉด์„œ๋„, ์‚ฌ์ „ํ•™์Šต๋œ ์–‘๋ฐฉํ–ฅ ํ‘œํ˜„์„ ๋งŽ์€ ๋ฌธ์ œ๋“ค์— ๋งž๊ฒŒ
๋ฏธ์„ธ์กฐ์ •ํ•˜์—ฌ ์ตœ๊ณ ์˜ ์„ฑ๋Šฅ์„ ์–ป๋Š” ๊ฒƒ์ž…๋‹ˆ๋‹ค. ์ด ํŠœํ† ๋ฆฌ์–ผ์—์„œ๋Š” ์‚ฌ์ „ํ•™์Šต๋œ
BERT ๋ชจ๋ธ์„ MRPC ๋ฌธ์ œ์— ๋งž๊ฒŒ ๋ฏธ์„ธ์กฐ์ •ํ•˜์—ฌ ์˜๋ฏธ์ ์œผ๋กœ ๋™์ผํ•œ ๋ฌธ์žฅ์„
๋ถ„๋ฅ˜ํ•ด๋ณด๊ฒ ์Šต๋‹ˆ๋‹ค.
๋ถ„๋ฅ˜ํ•ด ๋ณด๊ฒ ์Šต๋‹ˆ๋‹ค.

์‚ฌ์ „ํ•™์Šต๋œ BERT ๋ชจ๋ธ(HuggingFace transformer๋“ค ์ค‘ ``bert-base-uncased`` ๋ชจ๋ธ)์„
MRPC ๋ฌธ์ œ์— ๋งž๊ฒŒ ๋ฏธ์„ธ์กฐ์ •ํ•˜๊ธฐ ์œ„ํ•ด `์˜ˆ์‹œ๋“ค <https://github.com/huggingface/transformers/tree/master/examples#mrpc>`_
Expand Down Expand Up @@ -433,7 +433,7 @@ HuggingFace BERT ๋ชจ๋ธ์— ๋™์  ์–‘์žํ™”๋ฅผ ์ ์šฉํ•˜๊ธฐ ์œ„ํ•ด
3.1 ๋ชจ๋ธ ํฌ๊ธฐ ํ™•์ธํ•˜๊ธฐ
^^^^^^^^^^^^^^^^^^^^^^^^

๋จผ์ € ๋ชจ๋ธ ํฌ๊ธฐ๋ฅผ ํ™•์ธํ•ด๋ณด๊ฒ ์Šต๋‹ˆ๋‹ค. ๋ณด๋ฉด, ๋ชจ๋ธ ํฌ๊ธฐ๊ฐ€ ์ƒ๋‹นํžˆ ์ค„์–ด๋“  ๊ฒƒ์„
๋จผ์ € ๋ชจ๋ธ ํฌ๊ธฐ๋ฅผ ํ™•์ธํ•ด ๋ณด๊ฒ ์Šต๋‹ˆ๋‹ค. ๋ณด๋ฉด, ๋ชจ๋ธ ํฌ๊ธฐ๊ฐ€ ์ƒ๋‹นํžˆ ์ค„์–ด๋“  ๊ฒƒ์„
์•Œ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค(FP32 ํ˜•์‹์˜ ๋ชจ๋ธ ํฌ๊ธฐ : 438MB; INT8 ํ˜•์‹์˜ ๋ชจ๋ธ ํฌ๊ธฐ : 181MB):

.. code:: python
Expand All @@ -458,7 +458,7 @@ HuggingFace BERT ๋ชจ๋ธ์— ๋™์  ์–‘์žํ™”๋ฅผ ์ ์šฉํ•˜๊ธฐ ์œ„ํ•ด
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

๋‹ค์Œ์œผ๋กœ, ๊ธฐ์กด์˜ FP32 ๋ชจ๋ธ๊ณผ ๋™์  ์–‘์žํ™”๋ฅผ ์ ์šฉํ•œ INT8 ๋ชจ๋ธ๋“ค์˜
์ถ”๋ก  ์†๋„์™€ ์ •ํ™•๋„๋ฅผ ๋น„๊ตํ•ด๋ณด๊ฒŸ์Šต๋‹ˆ๋‹ค.
์ถ”๋ก  ์†๋„์™€ ์ •ํ™•๋„๋ฅผ ๋น„๊ตํ•ด ๋ณด๊ฒ ์Šต๋‹ˆ๋‹ค.

.. code:: python
Expand Down Expand Up @@ -492,7 +492,7 @@ HuggingFace BERT ๋ชจ๋ธ์— ๋™์  ์–‘์žํ™”๋ฅผ ์ ์šฉํ•˜๊ธฐ ์œ„ํ•ด
MRPC ๋ฌธ์ œ์— ๋งž๊ฒŒ ๋ฏธ์„ธ์กฐ์ •ํ•œ BERT ๋ชจ๋ธ์— ํ•™์Šต ํ›„ ๋™์  ์–‘์žํ™”๋ฅผ ์ ์šฉํ•œ
๊ฒฐ๊ณผ, 0.6% ๋‚ฎ์€ F1 ์ ์ˆ˜๊ฐ€ ๋‚˜์™”์Šต๋‹ˆ๋‹ค. ์ฐธ๊ณ ๋กœ, `์ตœ๊ทผ ๋…ผ๋ฌธ <https://arxiv.org/pdf/1910.06188.pdf>`_
(ํ‘œ 1)์—์„œ๋Š” ํ•™์Šต ํ›„ ๋™์  ์–‘์žํ™”๋ฅผ ์ ์šฉํ–ˆ์„ ๋•Œ, F1 ์ ์ˆ˜ 0.8788์ด ๋‚˜์™”๊ณ ,
์–‘์žํ™” ์˜์‹ ํ•™์Šต์„ ์ ์šฉํ–ˆ์„ ๋•Œ๋Š” 0.8956์ด ๋‚˜์™”์Šต๋‹ˆ๋‹ค. ์šฐ๋ฆฌ๋Š” Pytorch์˜ ๋น„๋Œ€์นญ
์–‘์žํ™” ์˜์‹ ํ•™์Šต์„ ์ ์šฉํ–ˆ์„ ๋•Œ๋Š” 0.8956์ด ๋‚˜์™”์Šต๋‹ˆ๋‹ค. ์šฐ๋ฆฌ๋Š” PyTorch์˜ ๋น„๋Œ€์นญ
์–‘์žํ™”๋ฅผ ์‚ฌ์šฉํ–ˆ์ง€๋งŒ, ์ฐธ๊ณ ํ•œ ๋…ผ๋ฌธ์—์„œ๋Š” ๋Œ€์นญ์  ์–‘์žํ™”๋งŒ์„ ์‚ฌ์šฉํ–ˆ๋‹ค๋Š” ์ ์ด ์ฃผ์š”ํ•œ
์ฐจ์ด์ž…๋‹ˆ๋‹ค.

Expand Down

0 comments on commit 0a68544

Please sign in to comment.