Skip to content

Commit e1ac5ed

Browse files
committed
Update readme.
1 parent 1f8904f commit e1ac5ed

File tree

1 file changed

+12
-22
lines changed

1 file changed

+12
-22
lines changed

libs/oci/README.md

Lines changed: 12 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -59,14 +59,16 @@ embeddings.embed_query("What is the meaning of life?")
5959

6060
### 1. Use a Chat Model
6161

62-
You may instantiate the OCI Data Science model with the generic `ChatOCIModelDeployment` or framework specific class like `ChatOCIModelDeploymentVLLM`.
62+
The `ChatOCIModelDeployment` class is designed for deployment with OpenAI compatible APIs.
6363

6464
```python
65-
from langchain_oci.chat_models import ChatOCIModelDeployment, ChatOCIModelDeploymentVLLM
65+
from langchain_oci import ChatOCIModelDeployment
6666

6767
# Create an instance of OCI Model Deployment Endpoint
68+
6869
# Replace the endpoint uri with your own
69-
endpoint = "https://modeldeployment.<region>.oci.customer-oci.com/<ocid>/predict"
70+
# For streaming, use the /predictWithResponseStream endpoint.
71+
endpoint = "https://modeldeployment.<region>.oci.customer-oci.com/<ocid>/predictWithResponseStream"
7072

7173
messages = [
7274
(
@@ -78,31 +80,23 @@ messages = [
7880

7981
chat = ChatOCIModelDeployment(
8082
endpoint=endpoint,
81-
streaming=True,
82-
max_retries=1,
83-
model_kwargs={
84-
"temperature": 0.2,
85-
"max_tokens": 512,
86-
}, # other model params...
87-
default_headers={
88-
"route": "/v1/chat/completions",
89-
# other request headers ...
90-
},
83+
model="odsc-llm",
84+
max_tokens=512,
9185
)
92-
chat.invoke(messages)
86+
chat.stream(messages)
9387

94-
chat_vllm = ChatOCIModelDeploymentVLLM(endpoint=endpoint)
95-
chat_vllm.invoke(messages)
9688
```
9789

9890
### 2. Use a Completion Model
99-
You may instantiate the OCI Data Science model with `OCIModelDeploymentLLM` or `OCIModelDeploymentVLLM`.
91+
The `OCIModelDeploymentLLM` class is designed for completion endpoints.
10092

10193
```python
102-
from langchain_oci.llms import OCIModelDeploymentLLM, OCIModelDeploymentVLLM
94+
from langchain_oci import OCIModelDeploymentLLM
10395

10496
# Create an instance of OCI Model Deployment Endpoint
97+
10598
# Replace the endpoint uri and model name with your own
99+
# For streaming, use the /predictWithResponseStream endpoint.
106100
endpoint = "https://modeldeployment.<region>.oci.customer-oci.com/<ocid>/predict"
107101

108102
llm = OCIModelDeploymentLLM(
@@ -111,10 +105,6 @@ llm = OCIModelDeploymentLLM(
111105
)
112106
llm.invoke("Who is the first president of United States?")
113107

114-
vllm = OCIModelDeploymentVLLM(
115-
endpoint=endpoint,
116-
)
117-
vllm.invoke("Who is the first president of United States?")
118108
```
119109

120110
### 3. Use an Embedding Model

0 commit comments

Comments
 (0)