Skip to content

Commit

Permalink
updated to include gpu dependency and quantization packages
Browse files Browse the repository at this point in the history
  • Loading branch information
samuel100 authored Nov 20, 2024
1 parent 37ec77f commit e70cfa2
Showing 1 changed file with 4 additions and 0 deletions.
4 changes: 4 additions & 0 deletions src/routes/blogs/olive-quant-ft/+page.svx
Original file line number Diff line number Diff line change
Expand Up @@ -41,11 +41,15 @@ Also, as part of answering the question of when to quantize we'll show how the f

To answer our question on the right sequencing of quantization and fine-tuning we leveraged Olive (ONNX Live) - an advanced model optimization toolkit designed to streamline the process of optimizing AI models for deployment with the ONNX runtime.

> **Note**: Both quantization and fine-tuning need to run on an Nvidia A10 or A100 GPU machine.

### 1. 💾 Install Olive

We installed the [Olive CLI](../blogs/olive-cli) using `pip`:

<pre><code>pip install olive-ai[quantize,finetuning]
pip install autoawq
pip install auto-gptq
</code></pre>

### 2. 🗜️ Quantize
Expand Down

0 comments on commit e70cfa2

Please sign in to comment.