-
Notifications
You must be signed in to change notification settings - Fork 0
chore: sync with text-to-cypher v0.1.19 and add token-usage example #104
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -142,7 +142,7 @@ console.log('Relationships:', schemaObj.relationships); | |
|
|
||
| Lists all available AI models across all supported providers. | ||
|
|
||
| **Note:** This method queries all provider APIs (OpenAI, Anthropic, Gemini, Ollama) and aggregates the results. Models are returned with provider prefixes (except OpenAI). | ||
| **Note:** This method queries all provider APIs (OpenAI, Anthropic, Gemini, Ollama) and aggregates the results. Each provider's live results are merged with a small curated static catalog, so providers with a curated list still return their well-known models even when no matching API key is configured. Providers without a curated list (e.g. Ollama) only appear when reachable. Models are returned with provider prefixes (except OpenAI). | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fixed in 11fa9bf — the README example output now uses the |
||
|
|
||
| **Returns:** `Promise<string[]>` - Array of model names with provider prefixes where applicable | ||
|
|
||
|
|
@@ -152,17 +152,22 @@ const models = await client.listModels(); | |
| console.log('All available models:', models); | ||
| // Output: [ | ||
| // 'gpt-4o-mini', 'gpt-4o', 'gpt-4-turbo', // OpenAI (no prefix) | ||
| // 'anthropic:claude-sonnet-4-5', 'anthropic:claude-opus-4-5', // Anthropic | ||
| // 'gemini:gemini-2.5-pro', 'gemini:gemini-3-pro-preview', // Gemini | ||
| // 'ollama:llama3', 'ollama:mistral' // Ollama (if running) | ||
| // 'anthropic::claude-sonnet-4-5', 'anthropic::claude-opus-4-5', // Anthropic | ||
| // 'gemini::gemini-2.5-pro', 'gemini::gemini-3-pro-preview', // Gemini | ||
| // 'ollama::llama3', 'ollama::mistral' // Ollama (if running) | ||
| // ] | ||
| ``` | ||
|
|
||
| Non-OpenAI models are namespaced with a `provider::model` (double-colon) prefix in the | ||
| returned list. When passing a model to the constructor you can use either that form or the | ||
| shorter `provider:model` (single-colon) form — both are accepted (e.g. | ||
| `'anthropic::claude-sonnet-4-5'` and `'anthropic:claude-sonnet-4-5'` are equivalent). | ||
|
|
||
| ### `listModelsByProvider(provider)` | ||
|
|
||
| Lists available AI models from a specific provider by querying the actual provider APIs. | ||
|
|
||
| **Note:** This method makes real API calls to the AI providers to get current model listings. The availability depends on your API credentials and the current offerings from each provider. | ||
| **Note:** This method merges each provider's live model list with a small curated static catalog. Providers with a curated list (OpenAI, Anthropic, Gemini) still return their well-known models even when no matching API key is configured; an API key adds any additional models the provider reports live. Providers without a curated list (e.g. Ollama) are only returned when reachable. | ||
|
|
||
| **Parameters:** | ||
| - `provider` (string): Provider name - `'openai'`, `'anthropic'`, `'gemini'`, or `'ollama'` (case-insensitive) | ||
|
|
@@ -232,6 +237,8 @@ answer, self-healing retries, and skill tool-call rounds). It is present on succ | |
| responses and omitted when no tokens were consumed. Failed requests reject with an error, | ||
| so `tokenUsage` is not surfaced for failures. | ||
|
|
||
| See [examples/token-usage.js](examples/token-usage.js) for a complete working example. | ||
|
|
||
| ### Message | ||
|
|
||
| ```typescript | ||
|
|
@@ -314,6 +321,24 @@ try { | |
| } | ||
| ``` | ||
|
|
||
| ### Tracking Token Usage | ||
|
|
||
| Each request aggregates the token counts from every LLM call it makes (cypher generation, | ||
| self-healing retries, skill tool-call rounds, and final answer generation) into | ||
| `response.tokenUsage`: | ||
|
|
||
| ```javascript | ||
| const response = await client.textToCypher('movies', 'How many actors are there?'); | ||
|
|
||
| if (response.tokenUsage) { | ||
| console.log('Prompt tokens:', response.tokenUsage.promptTokens); | ||
| console.log('Completion tokens:', response.tokenUsage.completionTokens); | ||
| console.log('Total tokens:', response.tokenUsage.totalTokens); | ||
| } | ||
| ``` | ||
|
|
||
| See [examples/token-usage.js](examples/token-usage.js) for a complete, runnable example. | ||
|
|
||
| ## Requirements | ||
|
|
||
| - Node.js >= 20 | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -199,6 +199,34 @@ describe('TextToCypher', () => { | |
| client.listModelsByProvider('invalid-provider') | ||
| ).rejects.toThrow(/Unknown provider/); | ||
| }); | ||
|
|
||
| // text-to-cypher >= 0.1.19 merges each provider's live list with a curated static | ||
| // catalog, so catalog-backed providers return well-known models even without an API key. | ||
| it('should return curated models without an API key', async () => { | ||
| const noKeyClient = new TextToCypher({ | ||
| model: 'gpt-4o-mini', | ||
| apiKey: '', | ||
| falkordbConnection: 'falkor://localhost:6379' | ||
| }); | ||
| for (const provider of ['openai', 'anthropic', 'gemini']) { | ||
| const models = await noKeyClient.listModelsByProvider(provider); | ||
| expect(Array.isArray(models)).toBe(true); | ||
| expect(models.length).toBeGreaterThan(0); | ||
| } | ||
| }, 30000); | ||
|
Comment on lines
+205
to
+216
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Good catch — both tests now construct a client with an empty |
||
|
|
||
| it('should aggregate curated models across providers without an API key', async () => { | ||
| const noKeyClient = new TextToCypher({ | ||
| model: 'gpt-4o-mini', | ||
| apiKey: '', | ||
| falkordbConnection: 'falkor://localhost:6379' | ||
| }); | ||
| const models = await noKeyClient.listModels(); | ||
| expect(Array.isArray(models)).toBe(true); | ||
| expect(models.length).toBeGreaterThan(0); | ||
| // Non-OpenAI providers are namespaced with a provider prefix. | ||
| expect(models.some(model => model.includes('::'))).toBe(true); | ||
| }, 30000); | ||
|
Comment on lines
+218
to
+229
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Good catch — both tests now construct a client with an empty |
||
| }); | ||
|
|
||
| describe('TokenUsage', () => { | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,103 @@ | ||
| /** | ||
| * Token-usage tracking example for @falkordb/text-to-cypher | ||
| * | ||
| * A single `textToCypher` / `cypherOnly` request may issue several LLM calls | ||
| * (schema-aware Cypher generation, optional self-healing retries, skill tool-call | ||
| * rounds, and final answer generation). The library aggregates the token counts | ||
| * from every one of those calls into `response.tokenUsage`. | ||
| * | ||
| * This example runs real requests against an AI model and prints the aggregated | ||
| * `tokenUsage` for both the full pipeline and the generation-only path. | ||
| * | ||
| * To run this example: | ||
| * 1. Ensure FalkorDB is running, e.g.: | ||
| * docker run -d -p 6379:6379 falkordb/falkordb:latest | ||
| * 2. Seed a graph so schema discovery returns a non-empty ontology, e.g.: | ||
| * redis-cli GRAPH.QUERY demo_graph \ | ||
| * "MERGE (:Person {name:'Alice'}) MERGE (:Person {name:'Bob'}) MERGE (:Person {name:'Charlie'})" | ||
| * 3. Provide an API key and (optionally) overrides via the environment, then run: | ||
| * OPENAI_API_KEY=sk-... node examples/token-usage.js | ||
| * or, using a .env file (Node >= 20.6): | ||
| * node --env-file=.env examples/token-usage.js | ||
| * | ||
| * Supported environment variables: | ||
| * OPENAI_API_KEY / DEFAULT_KEY - API key forwarded to the provider (required) | ||
| * MODEL / DEFAULT_MODEL - model to use (defaults to 'gpt-4o-mini') | ||
| * FALKORDB_CONNECTION - connection string (defaults to 'falkor://localhost:6379') | ||
| * GRAPH_NAME - graph to query (defaults to 'demo_graph') | ||
|
Comment on lines
+23
to
+27
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done — changed the default model to |
||
| */ | ||
|
|
||
| const { TextToCypher } = require('../index'); | ||
|
|
||
| function reportUsage(response) { | ||
| if (response.answer) { | ||
| console.log('Answer:', response.answer); | ||
| } | ||
|
|
||
| const usage = response.tokenUsage; | ||
| if (!usage) { | ||
| console.error( | ||
| '✗ No tokenUsage reported on the response — the provider may not return usage data.' | ||
| ); | ||
| return; | ||
| } | ||
|
|
||
| console.log('Token usage:'); | ||
| console.log(` promptTokens = ${usage.promptTokens}`); | ||
| console.log(` completionTokens = ${usage.completionTokens}`); | ||
| console.log(` totalTokens = ${usage.totalTokens}`); | ||
|
|
||
| if (usage.totalTokens === 0) { | ||
| console.error( | ||
| '⚠ tokenUsage was reported but totalTokens is 0 — the provider may not return usage data.' | ||
| ); | ||
| } else { | ||
| console.log('✓ Token usage tracking is working.'); | ||
| } | ||
| } | ||
|
|
||
| async function main() { | ||
| const apiKey = process.env.OPENAI_API_KEY || process.env.DEFAULT_KEY; | ||
| if (!apiKey) { | ||
| console.error('Please set OPENAI_API_KEY (or DEFAULT_KEY) to run this example.'); | ||
| process.exit(1); | ||
| } | ||
|
|
||
| const model = process.env.MODEL || process.env.DEFAULT_MODEL || 'gpt-4o-mini'; | ||
| const falkordbConnection = process.env.FALKORDB_CONNECTION || 'falkor://localhost:6379'; | ||
| const graphName = process.env.GRAPH_NAME || 'demo_graph'; | ||
|
|
||
| console.log('=== Token Usage Tracking Example ==='); | ||
| console.log(`model: ${model}`); | ||
| console.log(`connection: ${falkordbConnection}`); | ||
| console.log(`graph: ${graphName}\n`); | ||
|
|
||
| const client = new TextToCypher({ model, apiKey, falkordbConnection }); | ||
|
|
||
| // Example 1: full pipeline (generate -> execute -> answer). Usage is summed across | ||
| // every LLM call the request made. | ||
| console.log('--- Example 1: full textToCypher pipeline ---'); | ||
| try { | ||
| const response = await client.textToCypher(graphName, 'How many people are in the graph?'); | ||
| reportUsage(response); | ||
| } catch (error) { | ||
| console.error('✗ Request failed:', error.message); | ||
| } | ||
|
|
||
| // Example 2: cypherOnly (no execution / no answer generation). Usage should reflect | ||
| // just the query-generation call(s) and therefore typically be smaller. | ||
| console.log('\n--- Example 2: cypherOnly (generation only) ---'); | ||
| try { | ||
| const response = await client.cypherOnly(graphName, 'List the names of all people.'); | ||
| if (response.cypherQuery) { | ||
| console.log('Generated query:', response.cypherQuery); | ||
| } | ||
| reportUsage(response); | ||
| } catch (error) { | ||
| console.error('✗ Request failed:', error.message); | ||
| } | ||
|
|
||
| console.log('\n=== Example completed ==='); | ||
| } | ||
|
|
||
| main().catch(console.error); | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed in 11fa9bf — the README example output now uses the
provider::model(double-colon) form thatlistModels()actually returns, and I added a note clarifying that the constructor accepts bothprovider::modeland the shorterprovider:modelform (normalize_model_namein src/lib.rs converts the latter to the former).