Skip to content

Commit

Permalink
openai: add support for setting the base url
Browse files Browse the repository at this point in the history
This commit adds support for setting the base url for the openai
client. This is useful for testing and for using the client with
a different provider.

The motivation for this came from wanting to try out Perplexities API
using `llm-chain-openai`. For this I needed to set the base url to
the one provided by Perplexity, but I could find a way to that with
the current implementation.

With the change in this commit, an example can be written like this:
```rust
use llm_chain::options;
use llm_chain::options::ModelRef;
use llm_chain::{executor, parameters, prompt};

async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let opts = options!(
         Model: ModelRef::from_model_name("pplx-70b-online")
    );
    let exec = executor!(chatgpt, opts.clone())?;
    let query = "What is the capital of Sweden?";
    println!("Query: {query}\n");
    let res = prompt!("", query,).run(&parameters!(), &exec).await?;
    println!("Perplixity AI:\n{res}");
    Ok(())
}
```
And the following environment variables need to be set:
```console
export OPENAI_API_KEY=<Perplexity-API-Key>
export OPENAI_API_BASE_URL=https://api.perplexity.ai
```

Signed-off-by: Daniel Bevenius <[email protected]>
  • Loading branch information
danbev committed Feb 3, 2024
1 parent a416579 commit 2ecfbff
Showing 1 changed file with 5 additions and 0 deletions.
5 changes: 5 additions & 0 deletions crates/llm-chain-openai/src/chatgpt/executor.rs
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,11 @@ impl traits::Executor for Executor {
if let Ok(org_id) = std::env::var("OPENAI_ORG_ID") {
cfg = cfg.with_org_id(org_id);
}

if let Ok(base_url) = std::env::var("OPENAI_API_BASE_URL") {
cfg = cfg.with_api_base(base_url);
}

let client = Arc::new(async_openai::Client::with_config(cfg));
Ok(Self { client, options })
}
Expand Down

0 comments on commit 2ecfbff

Please sign in to comment.