-
Notifications
You must be signed in to change notification settings - Fork 375
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
58 changed files
with
1,368 additions
and
79 deletions.
There are no files selected for viewing
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
--- | ||
title: Agents | ||
--- | ||
|
||
A built-in agent that can take decisions and reasoning based on the tools provided to it. | ||
|
||
## OpenAI Agent | ||
|
||
import { DynamicCodeBlock } from 'fumadocs-ui/components/dynamic-codeblock'; | ||
import CodeSource from "!raw-loader!../../../../../../../examples/agent/openai"; | ||
|
||
<DynamicCodeBlock lang="ts" code={CodeSource} /> |
8 changes: 8 additions & 0 deletions
8
apps/next/src/content/docs/llamaindex/examples/agent_gemini.mdx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
--- | ||
title: Gemini Agent | ||
--- | ||
|
||
import { DynamicCodeBlock } from 'fumadocs-ui/components/dynamic-codeblock'; | ||
import CodeSourceGemini from "!raw-loader!../../../../../../../examples/gemini/agent.ts"; | ||
|
||
<DynamicCodeBlock lang="ts" code={CodeSourceGemini} /> |
10 changes: 10 additions & 0 deletions
10
apps/next/src/content/docs/llamaindex/examples/chat_engine.mdx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
--- | ||
title: Chat Engine | ||
--- | ||
|
||
import { DynamicCodeBlock } from 'fumadocs-ui/components/dynamic-codeblock'; | ||
import CodeSource from "!raw-loader!../../../../../../../examples/chatEngine"; | ||
|
||
Chat Engine is a class that allows you to create a chatbot from a retriever. It is a wrapper around a retriever that allows you to chat with it in a conversational manner. | ||
|
||
<DynamicCodeBlock lang="ts" code={CodeSource} /> |
61 changes: 61 additions & 0 deletions
61
apps/next/src/content/docs/llamaindex/examples/context_aware_agent.mdx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,61 @@ | ||
--- | ||
title: Context-Aware Agent | ||
--- | ||
|
||
The Context-Aware Agent enhances the capabilities of standard LLM agents by incorporating relevant context from a retriever for each query. This allows the agent to provide more informed and specific responses based on the available information. | ||
|
||
## Usage | ||
|
||
Here's a simple example of how to use the Context-Aware Agent: | ||
|
||
```typescript | ||
import { | ||
Document, | ||
VectorStoreIndex, | ||
OpenAIContextAwareAgent, | ||
OpenAI, | ||
} from "llamaindex"; | ||
|
||
async function createContextAwareAgent() { | ||
// Create and index some documents | ||
const documents = [ | ||
new Document({ | ||
text: "LlamaIndex is a data framework for LLM applications.", | ||
id_: "doc1", | ||
}), | ||
new Document({ | ||
text: "The Eiffel Tower is located in Paris, France.", | ||
id_: "doc2", | ||
}), | ||
]; | ||
|
||
const index = await VectorStoreIndex.fromDocuments(documents); | ||
const retriever = index.asRetriever({ similarityTopK: 1 }); | ||
|
||
// Create the Context-Aware Agent | ||
const agent = new OpenAIContextAwareAgent({ | ||
llm: new OpenAI({ model: "gpt-3.5-turbo" }), | ||
contextRetriever: retriever, | ||
}); | ||
|
||
// Use the agent to answer queries | ||
const response = await agent.chat({ | ||
message: "What is LlamaIndex used for?", | ||
}); | ||
|
||
console.log("Agent Response:", response.response); | ||
} | ||
|
||
createContextAwareAgent().catch(console.error); | ||
``` | ||
|
||
In this example, the Context-Aware Agent uses the retriever to fetch relevant context for each query, allowing it to provide more accurate and informed responses based on the indexed documents. | ||
|
||
## Key Components | ||
|
||
- `contextRetriever`: A retriever (e.g., from a VectorStoreIndex) that fetches relevant documents or passages for each query. | ||
|
||
## Available Context-Aware Agents | ||
|
||
- `OpenAIContextAwareAgent`: A context-aware agent using OpenAI's models. | ||
- `AnthropicContextAwareAgent`: A context-aware agent using Anthropic's models. |
79 changes: 79 additions & 0 deletions
79
apps/next/src/content/docs/llamaindex/examples/local_llm.mdx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,79 @@ | ||
--- | ||
title: Local LLMs | ||
--- | ||
|
||
LlamaIndex.TS supports OpenAI and [other remote LLM APIs](other_llms). You can also run a local LLM on your machine! | ||
|
||
## Using a local model via Ollama | ||
|
||
The easiest way to run a local LLM is via the great work of our friends at [Ollama](https://ollama.com/), who provide a simple to use client that will download, install and run a [growing range of models](https://ollama.com/library) for you. | ||
|
||
### Install Ollama | ||
|
||
They provide a one-click installer for Mac, Linux and Windows on their [home page](https://ollama.com/). | ||
|
||
### Pick and run a model | ||
|
||
Since we're going to be doing agentic work, we'll need a very capable model, but the largest models are hard to run on a laptop. We think `mixtral 8x7b` is a good balance between power and resources, but `llama3` is another great option. You can run Mixtral by running | ||
|
||
```bash | ||
ollama run mixtral:8x7b | ||
``` | ||
|
||
The first time you run it will also automatically download and install the model for you. | ||
|
||
### Switch the LLM in your code | ||
|
||
To tell LlamaIndex to use a local LLM, use the `Settings` object: | ||
|
||
```javascript | ||
Settings.llm = new Ollama({ | ||
model: "mixtral:8x7b", | ||
}); | ||
``` | ||
|
||
### Use local embeddings | ||
|
||
If you're doing retrieval-augmented generation, LlamaIndex.TS will also call out to OpenAI to index and embed your data. To be entirely local, you can use a local embedding model like this: | ||
|
||
```javascript | ||
Settings.embedModel = new HuggingFaceEmbedding({ | ||
modelType: "BAAI/bge-small-en-v1.5", | ||
quantized: false, | ||
}); | ||
``` | ||
|
||
The first time this runs it will download the embedding model to run it. | ||
|
||
### Try it out | ||
|
||
With a local LLM and local embeddings in place, you can perform RAG as usual and everything will happen on your machine without calling an API: | ||
|
||
```typescript | ||
async function main() { | ||
// Load essay from abramov.txt in Node | ||
const path = "node_modules/llamaindex/examples/abramov.txt"; | ||
|
||
const essay = await fs.readFile(path, "utf-8"); | ||
|
||
// Create Document object with essay | ||
const document = new Document({ text: essay, id_: path }); | ||
|
||
// Split text and create embeddings. Store them in a VectorStoreIndex | ||
const index = await VectorStoreIndex.fromDocuments([document]); | ||
|
||
// Query the index | ||
const queryEngine = index.asQueryEngine(); | ||
|
||
const response = await queryEngine.query({ | ||
query: "What did the author do in college?", | ||
}); | ||
|
||
// Output response | ||
console.log(response.toString()); | ||
} | ||
|
||
main().catch(console.error); | ||
``` | ||
|
||
You can see the [full example file](https://github.com/run-llama/LlamaIndexTS/blob/main/examples/vectorIndexLocal.ts). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
{ | ||
"title": "Examples", | ||
"pages": [ | ||
"more_examples", | ||
"chat_engine", | ||
"vector_index", | ||
"summary_index", | ||
"save_load_index", | ||
"context_aware_agent", | ||
"agent", | ||
"agent_gemini", | ||
"local_llm", | ||
"other_llms" | ||
] | ||
} |
21 changes: 21 additions & 0 deletions
21
apps/next/src/content/docs/llamaindex/examples/more_examples.mdx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
--- | ||
title: See all examples | ||
--- | ||
|
||
Our GitHub repository has a wealth of examples to explore and try out. You can check out our [examples folder](https://github.com/run-llama/LlamaIndexTS/tree/main/examples) to see them all at once, or browse the pages in this section for some selected highlights. | ||
|
||
## Check out all examples | ||
|
||
It may be useful to check out all the examples at once so you can try them out locally. To do this into a folder called `my-new-project`, run these commands: | ||
|
||
```bash npm2yarn | ||
npx degit run-llama/LlamaIndexTS/examples my-new-project | ||
cd my-new-project | ||
npm install | ||
``` | ||
|
||
Then you can run any example in the folder with `tsx`, e.g.: | ||
|
||
```bash npm2yarn | ||
npx tsx ./vectorIndex.ts | ||
``` |
43 changes: 43 additions & 0 deletions
43
apps/next/src/content/docs/llamaindex/examples/other_llms.mdx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,43 @@ | ||
--- | ||
title: Using other LLM APIs | ||
--- | ||
|
||
import { DynamicCodeBlock } from 'fumadocs-ui/components/dynamic-codeblock'; | ||
import CodeSource from "!raw-loader!../../../../../../../examples/mistral"; | ||
|
||
By default LlamaIndex.TS uses OpenAI's LLMs and embedding models, but we support [lots of other LLMs](../modules/llms) including models from Mistral (Mistral, Mixtral), Anthropic (Claude) and Google (Gemini). | ||
|
||
If you don't want to use an API at all you can [run a local model](../../examples/local_llm) | ||
|
||
## Using another LLM | ||
|
||
You can specify what LLM LlamaIndex.TS will use on the `Settings` object, like this: | ||
|
||
```typescript | ||
import { MistralAI, Settings } from "llamaindex"; | ||
|
||
Settings.llm = new MistralAI({ | ||
model: "mistral-tiny", | ||
apiKey: "<YOUR_API_KEY>", | ||
}); | ||
``` | ||
|
||
You can see examples of other APIs we support by checking out "Available LLMs" in the sidebar of our [LLMs section](../modules/llms). | ||
|
||
## Using another embedding model | ||
|
||
A frequent gotcha when trying to use a different API as your LLM is that LlamaIndex will also by default index and embed your data using OpenAI's embeddings. To completely switch away from OpenAI you will need to set your embedding model as well, for example: | ||
|
||
```typescript | ||
import { MistralAIEmbedding, Settings } from "llamaindex"; | ||
|
||
Settings.embedModel = new MistralAIEmbedding(); | ||
``` | ||
|
||
We support [many different embeddings](../modules/embeddings). | ||
|
||
## Full example | ||
|
||
This example uses Mistral's `mistral-tiny` model as the LLM and Mistral for embeddings as well. | ||
|
||
<DynamicCodeBlock lang="ts" code={CodeSource} /> |
8 changes: 8 additions & 0 deletions
8
apps/next/src/content/docs/llamaindex/examples/save_load_index.mdx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
--- | ||
title: Save/Load an Index | ||
--- | ||
|
||
import { DynamicCodeBlock } from 'fumadocs-ui/components/dynamic-codeblock'; | ||
import CodeSource from "!raw-loader!../../../../../../../examples/storageContext"; | ||
|
||
<DynamicCodeBlock lang="ts" code={CodeSource} /> |
8 changes: 8 additions & 0 deletions
8
apps/next/src/content/docs/llamaindex/examples/summary_index.mdx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
--- | ||
title: Summary Index | ||
--- | ||
|
||
import { DynamicCodeBlock } from 'fumadocs-ui/components/dynamic-codeblock'; | ||
import CodeSource from "!raw-loader!../../../../../../../examples/summaryIndex"; | ||
|
||
<DynamicCodeBlock lang="ts" code={CodeSource} /> |
8 changes: 8 additions & 0 deletions
8
apps/next/src/content/docs/llamaindex/examples/vector_index.mdx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
--- | ||
title: Vector Index | ||
--- | ||
|
||
import { DynamicCodeBlock } from 'fumadocs-ui/components/dynamic-codeblock'; | ||
import CodeSource from "!raw-loader!../../../../../../../examples/vectorIndex"; | ||
|
||
<DynamicCodeBlock lang="ts" code={CodeSource} /> |
76 changes: 76 additions & 0 deletions
76
apps/next/src/content/docs/llamaindex/getting_started/concepts.mdx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,76 @@ | ||
--- | ||
title: Concepts | ||
--- | ||
|
||
LlamaIndex.TS helps you build LLM-powered applications (e.g. Q&A, chatbot) over custom data. | ||
|
||
In this high-level concepts guide, you will learn: | ||
|
||
- how an LLM can answer questions using your own data. | ||
- key concepts and modules in LlamaIndex.TS for composing your own query pipeline. | ||
|
||
## Answering Questions Across Your Data | ||
|
||
LlamaIndex uses a two stage method when using an LLM with your data: | ||
|
||
1. **indexing stage**: preparing a knowledge base, and | ||
2. **querying stage**: retrieving relevant context from the knowledge to assist the LLM in responding to a question | ||
|
||
![](../_static/concepts/rag.jpg) | ||
|
||
This process is also known as Retrieval Augmented Generation (RAG). | ||
|
||
LlamaIndex.TS provides the essential toolkit for making both steps super easy. | ||
|
||
Let's explore each stage in detail. | ||
|
||
### Indexing Stage | ||
|
||
LlamaIndex.TS help you prepare the knowledge base with a suite of data connectors and indexes. | ||
|
||
![](../_static/concepts/indexing.jpg) | ||
|
||
[**Data Loaders**](/docs/llamaindex/modules/data_loaders/index): | ||
A data connector (i.e. `Reader`) ingest data from different data sources and data formats into a simple `Document` representation (text and simple metadata). | ||
|
||
[**Documents / Nodes**](/docs/llamaindex/modules/documents_and_nodes/index): A `Document` is a generic container around any data source - for instance, a PDF, an API output, or retrieved data from a database. A `Node` is the atomic unit of data in LlamaIndex and represents a "chunk" of a source `Document`. It's a rich representation that includes metadata and relationships (to other nodes) to enable accurate and expressive retrieval operations. | ||
|
||
[**Data Indexes**](/docs/llamaindex/modules/data_index): | ||
Once you've ingested your data, LlamaIndex helps you index data into a format that's easy to retrieve. | ||
|
||
Under the hood, LlamaIndex parses the raw documents into intermediate representations, calculates vector embeddings, and stores your data in-memory or to disk. | ||
|
||
### Querying Stage | ||
|
||
In the querying stage, the query pipeline retrieves the most relevant context given a user query, | ||
and pass that to the LLM (along with the query) to synthesize a response. | ||
|
||
This gives the LLM up-to-date knowledge that is not in its original training data, | ||
(also reducing hallucination). | ||
|
||
The key challenge in the querying stage is retrieval, orchestration, and reasoning over (potentially many) knowledge bases. | ||
|
||
LlamaIndex provides composable modules that help you build and integrate RAG pipelines for Q&A (query engine), chatbot (chat engine), or as part of an agent. | ||
|
||
These building blocks can be customized to reflect ranking preferences, as well as composed to reason over multiple knowledge bases in a structured way. | ||
|
||
![](../_static/concepts/querying.jpg) | ||
|
||
#### Building Blocks | ||
|
||
[**Retrievers**](/docs/llamaindex/modules/retriever): | ||
A retriever defines how to efficiently retrieve relevant context from a knowledge base (i.e. index) when given a query. | ||
The specific retrieval logic differs for different indices, the most popular being dense retrieval against a vector index. | ||
|
||
[**Response Synthesizers**](/docs/llamaindex/modules/response_synthesizer): | ||
A response synthesizer generates a response from an LLM, using a user query and a given set of retrieved text chunks. | ||
|
||
#### Pipelines | ||
|
||
[**Query Engines**](/docs/llamaindex/modules/query_engines): | ||
A query engine is an end-to-end pipeline that allow you to ask question over your data. | ||
It takes in a natural language query, and returns a response, along with reference context retrieved and passed to the LLM. | ||
|
||
[**Chat Engines**](/docs/llamaindex/modules/chat_engine): | ||
A chat engine is an end-to-end pipeline for having a conversation with your data | ||
(multiple back-and-forth instead of a single question & answer). |
20 changes: 20 additions & 0 deletions
20
apps/next/src/content/docs/llamaindex/getting_started/environments.mdx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
--- | ||
title: Environments | ||
--- | ||
|
||
We support Node.JS versions 18, 20 and 22, with experimental support for Deno, Bun and Vercel Edge functions. | ||
|
||
## NextJS | ||
|
||
If you're using NextJS you'll need to add `withLlamaIndex` to your `next.config.js` file. This will add the necessary configuration for included 3rd-party libraries to your build: | ||
|
||
```js | ||
// next.config.js | ||
const withLlamaIndex = require("llamaindex/next"); | ||
|
||
module.exports = withLlamaIndex({ | ||
// your next.js config | ||
}); | ||
``` | ||
|
||
For details, check the latest [withLlamaIndex](https://github.com/run-llama/LlamaIndexTS/blob/main/packages/llamaindex/src/next.ts) implementation. |
Oops, something went wrong.