Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,21 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [0.1.16] - 2026-06-03

### Added
- New `examples/token-usage.js` example demonstrating how to read aggregated
`tokenUsage` (`promptTokens`, `completionTokens`, `totalTokens`) from responses for
both the full `textToCypher` pipeline and the generation-only `cypherOnly` path.
- README and examples documentation for token-usage tracking and a clearer note that
model discovery merges each provider's live list with a curated static catalog.

### Changed
- Bumped the `text-to-cypher` Rust dependency to `0.1.19`. `listModels()` /
`listModelsByProvider()` now merge live provider results with a curated static model
catalog, so providers with a curated list return their well-known models even when no
matching API key is configured.

## [0.1.15] - 2026-06-02

### Added
Expand Down Expand Up @@ -72,5 +87,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Proper error handling and propagation from Rust to JavaScript
- Zero runtime dependencies

[0.1.16]: https://github.com/FalkorDB/text-to-cypher-node/releases/tag/v0.1.16
[0.1.15]: https://github.com/FalkorDB/text-to-cypher-node/releases/tag/v0.1.15
[0.1.0]: https://github.com/FalkorDB/text-to-cypher-node/releases/tag/v0.1.0
6 changes: 3 additions & 3 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "text-to-cypher-node"
version = "0.1.15"
version = "0.1.16"
edition = "2021"
authors = ["FalkorDB"]
license = "MIT"
Expand All @@ -17,7 +17,7 @@ napi-derive = "3"
# Disable default features to avoid including server dependencies (actix-web, etc.)
# We only need the core library functionality for the bindings
# Explicitly set features to empty array to ensure no features are enabled
text-to-cypher = { version = "0.1.18", default-features = false, features = [] }
text-to-cypher = { version = "0.1.19", default-features = false, features = [] }
tokio = { version = "1", features = ["rt-multi-thread", "macros"] }
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
Expand Down
35 changes: 30 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@ console.log('Relationships:', schemaObj.relationships);

Lists all available AI models across all supported providers.

**Note:** This method queries all provider APIs (OpenAI, Anthropic, Gemini, Ollama) and aggregates the results. Models are returned with provider prefixes (except OpenAI).
**Note:** This method queries all provider APIs (OpenAI, Anthropic, Gemini, Ollama) and aggregates the results. Each provider's live results are merged with a small curated static catalog, so providers with a curated list still return their well-known models even when no matching API key is configured. Providers without a curated list (e.g. Ollama) only appear when reachable. Models are returned with provider prefixes (except OpenAI).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 11fa9bf — the README example output now uses the provider::model (double-colon) form that listModels() actually returns, and I added a note clarifying that the constructor accepts both provider::model and the shorter provider:model form (normalize_model_name in src/lib.rs converts the latter to the former).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 11fa9bf — the README example output now uses the provider::model (double-colon) form that listModels() actually returns, and I added a note clarifying that the constructor accepts both provider::model and the shorter provider:model form (normalize_model_name in src/lib.rs converts the latter to the former).


**Returns:** `Promise<string[]>` - Array of model names with provider prefixes where applicable

Expand All @@ -152,17 +152,22 @@ const models = await client.listModels();
console.log('All available models:', models);
// Output: [
// 'gpt-4o-mini', 'gpt-4o', 'gpt-4-turbo', // OpenAI (no prefix)
// 'anthropic:claude-sonnet-4-5', 'anthropic:claude-opus-4-5', // Anthropic
// 'gemini:gemini-2.5-pro', 'gemini:gemini-3-pro-preview', // Gemini
// 'ollama:llama3', 'ollama:mistral' // Ollama (if running)
// 'anthropic::claude-sonnet-4-5', 'anthropic::claude-opus-4-5', // Anthropic
// 'gemini::gemini-2.5-pro', 'gemini::gemini-3-pro-preview', // Gemini
// 'ollama::llama3', 'ollama::mistral' // Ollama (if running)
// ]
```

Non-OpenAI models are namespaced with a `provider::model` (double-colon) prefix in the
returned list. When passing a model to the constructor you can use either that form or the
shorter `provider:model` (single-colon) form — both are accepted (e.g.
`'anthropic::claude-sonnet-4-5'` and `'anthropic:claude-sonnet-4-5'` are equivalent).

### `listModelsByProvider(provider)`

Lists available AI models from a specific provider by querying the actual provider APIs.

**Note:** This method makes real API calls to the AI providers to get current model listings. The availability depends on your API credentials and the current offerings from each provider.
**Note:** This method merges each provider's live model list with a small curated static catalog. Providers with a curated list (OpenAI, Anthropic, Gemini) still return their well-known models even when no matching API key is configured; an API key adds any additional models the provider reports live. Providers without a curated list (e.g. Ollama) are only returned when reachable.

**Parameters:**
- `provider` (string): Provider name - `'openai'`, `'anthropic'`, `'gemini'`, or `'ollama'` (case-insensitive)
Expand Down Expand Up @@ -232,6 +237,8 @@ answer, self-healing retries, and skill tool-call rounds). It is present on succ
responses and omitted when no tokens were consumed. Failed requests reject with an error,
so `tokenUsage` is not surfaced for failures.

See [examples/token-usage.js](examples/token-usage.js) for a complete working example.

### Message

```typescript
Expand Down Expand Up @@ -314,6 +321,24 @@ try {
}
```

### Tracking Token Usage

Each request aggregates the token counts from every LLM call it makes (cypher generation,
self-healing retries, skill tool-call rounds, and final answer generation) into
`response.tokenUsage`:

```javascript
const response = await client.textToCypher('movies', 'How many actors are there?');

if (response.tokenUsage) {
console.log('Prompt tokens:', response.tokenUsage.promptTokens);
console.log('Completion tokens:', response.tokenUsage.completionTokens);
console.log('Total tokens:', response.tokenUsage.totalTokens);
}
```

See [examples/token-usage.js](examples/token-usage.js) for a complete, runnable example.

## Requirements

- Node.js >= 20
Expand Down
28 changes: 28 additions & 0 deletions __test__/index.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -199,6 +199,34 @@ describe('TextToCypher', () => {
client.listModelsByProvider('invalid-provider')
).rejects.toThrow(/Unknown provider/);
});

// text-to-cypher >= 0.1.19 merges each provider's live list with a curated static
// catalog, so catalog-backed providers return well-known models even without an API key.
it('should return curated models without an API key', async () => {
const noKeyClient = new TextToCypher({
model: 'gpt-4o-mini',
apiKey: '',
falkordbConnection: 'falkor://localhost:6379'
});
for (const provider of ['openai', 'anthropic', 'gemini']) {
const models = await noKeyClient.listModelsByProvider(provider);
expect(Array.isArray(models)).toBe(true);
expect(models.length).toBeGreaterThan(0);
}
}, 30000);
Comment on lines +205 to +216

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — both tests now construct a client with an empty apiKey so they genuinely exercise the no-key curated-catalog path instead of reusing the test-key client. Fixed in 1b11ea2.


it('should aggregate curated models across providers without an API key', async () => {
const noKeyClient = new TextToCypher({
model: 'gpt-4o-mini',
apiKey: '',
falkordbConnection: 'falkor://localhost:6379'
});
const models = await noKeyClient.listModels();
expect(Array.isArray(models)).toBe(true);
expect(models.length).toBeGreaterThan(0);
// Non-OpenAI providers are namespaced with a provider prefix.
expect(models.some(model => model.includes('::'))).toBe(true);
}, 30000);
Comment on lines +218 to +229

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — both tests now construct a client with an empty apiKey so they genuinely exercise the no-key curated-catalog path instead of reusing the test-key client. Fixed in 1b11ea2.

});

describe('TokenUsage', () => {
Expand Down
32 changes: 31 additions & 1 deletion examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,34 @@ export FALKORDB_CONNECTION="falkor://localhost:6379"
node examples/basic-usage.js
```

### Model Discovery Example

```bash
# An API key lets each provider return its live model list; a curated static
# catalog is also merged in, so well-known models appear even without a key.
export OPENAI_API_KEY="your-api-key"

node examples/list-models.js
```

### Token Usage Example

```bash
# Requires a running FalkorDB and a seeded graph so schema discovery succeeds.
docker run -d -p 6379:6379 falkordb/falkordb:latest
redis-cli GRAPH.QUERY demo_graph \
"MERGE (:Person {name:'Alice'}) MERGE (:Person {name:'Bob'}) MERGE (:Person {name:'Charlie'})"

# Provide an API key (the account must have available quota)
export OPENAI_API_KEY="sk-..."

# MODEL defaults to gpt-4o-mini; GRAPH_NAME defaults to demo_graph
node examples/token-usage.js

# Or load variables from a .env file (Node >= 20.6)
node --env-file=.env examples/token-usage.js
```

### TypeScript Example

```bash
Expand All @@ -28,7 +56,9 @@ ts-node examples/typescript-usage.ts
## Examples Included

1. **basic-usage.js** - Comprehensive JavaScript example showing all major features
2. **typescript-usage.ts** - TypeScript example with type safety
2. **list-models.js** - Model discovery across providers (`listModels` / `listModelsByProvider`)
3. **token-usage.js** - Reading aggregated LLM token usage from `response.tokenUsage`
4. **typescript-usage.ts** - TypeScript example with type safety

## Prerequisites

Expand Down
103 changes: 103 additions & 0 deletions examples/token-usage.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
/**
* Token-usage tracking example for @falkordb/text-to-cypher
*
* A single `textToCypher` / `cypherOnly` request may issue several LLM calls
* (schema-aware Cypher generation, optional self-healing retries, skill tool-call
* rounds, and final answer generation). The library aggregates the token counts
* from every one of those calls into `response.tokenUsage`.
*
* This example runs real requests against an AI model and prints the aggregated
* `tokenUsage` for both the full pipeline and the generation-only path.
*
* To run this example:
* 1. Ensure FalkorDB is running, e.g.:
* docker run -d -p 6379:6379 falkordb/falkordb:latest
* 2. Seed a graph so schema discovery returns a non-empty ontology, e.g.:
* redis-cli GRAPH.QUERY demo_graph \
* "MERGE (:Person {name:'Alice'}) MERGE (:Person {name:'Bob'}) MERGE (:Person {name:'Charlie'})"
* 3. Provide an API key and (optionally) overrides via the environment, then run:
* OPENAI_API_KEY=sk-... node examples/token-usage.js
* or, using a .env file (Node >= 20.6):
* node --env-file=.env examples/token-usage.js
*
* Supported environment variables:
* OPENAI_API_KEY / DEFAULT_KEY - API key forwarded to the provider (required)
* MODEL / DEFAULT_MODEL - model to use (defaults to 'gpt-4o-mini')
* FALKORDB_CONNECTION - connection string (defaults to 'falkor://localhost:6379')
* GRAPH_NAME - graph to query (defaults to 'demo_graph')
Comment on lines +23 to +27

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done — changed the default model to gpt-4o-mini (in the code, header comment, and examples README) for consistency with the rest of the repo. It still respects MODEL/DEFAULT_MODEL overrides, and gpt-4o-mini also reports token usage. Fixed in 1b11ea2.

*/

const { TextToCypher } = require('../index');

function reportUsage(response) {
if (response.answer) {
console.log('Answer:', response.answer);
}

const usage = response.tokenUsage;
if (!usage) {
console.error(
'✗ No tokenUsage reported on the response — the provider may not return usage data.'
);
return;
}

console.log('Token usage:');
console.log(` promptTokens = ${usage.promptTokens}`);
console.log(` completionTokens = ${usage.completionTokens}`);
console.log(` totalTokens = ${usage.totalTokens}`);

if (usage.totalTokens === 0) {
console.error(
'⚠ tokenUsage was reported but totalTokens is 0 — the provider may not return usage data.'
);
} else {
console.log('✓ Token usage tracking is working.');
}
}

async function main() {
const apiKey = process.env.OPENAI_API_KEY || process.env.DEFAULT_KEY;
if (!apiKey) {
console.error('Please set OPENAI_API_KEY (or DEFAULT_KEY) to run this example.');
process.exit(1);
}

const model = process.env.MODEL || process.env.DEFAULT_MODEL || 'gpt-4o-mini';
const falkordbConnection = process.env.FALKORDB_CONNECTION || 'falkor://localhost:6379';
const graphName = process.env.GRAPH_NAME || 'demo_graph';

console.log('=== Token Usage Tracking Example ===');
console.log(`model: ${model}`);
console.log(`connection: ${falkordbConnection}`);
console.log(`graph: ${graphName}\n`);

const client = new TextToCypher({ model, apiKey, falkordbConnection });

// Example 1: full pipeline (generate -> execute -> answer). Usage is summed across
// every LLM call the request made.
console.log('--- Example 1: full textToCypher pipeline ---');
try {
const response = await client.textToCypher(graphName, 'How many people are in the graph?');
reportUsage(response);
} catch (error) {
console.error('✗ Request failed:', error.message);
}

// Example 2: cypherOnly (no execution / no answer generation). Usage should reflect
// just the query-generation call(s) and therefore typically be smaller.
console.log('\n--- Example 2: cypherOnly (generation only) ---');
try {
const response = await client.cypherOnly(graphName, 'List the names of all people.');
if (response.cypherQuery) {
console.log('Generated query:', response.cypherQuery);
}
reportUsage(response);
} catch (error) {
console.error('✗ Request failed:', error.message);
}

console.log('\n=== Example completed ===');
}

main().catch(console.error);
6 changes: 4 additions & 2 deletions index.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -165,8 +165,10 @@ export declare class TextToCypher {
*
* # Note
*
* This method queries the actual AI provider APIs to get the list of available models.
* The availability depends on your API credentials and the current offerings from each provider.
* Each provider's live results are merged with a curated static catalog, so
* providers with a curated list (OpenAI, Anthropic, Gemini) still return their
* well-known models even when no matching API key is configured. Providers without
* a curated list (e.g. Ollama) are only returned when reachable.
*
* # Returns
*
Expand Down
6 changes: 4 additions & 2 deletions src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -370,8 +370,10 @@ impl TextToCypher {
///
/// # Note
///
/// This method queries the actual AI provider APIs to get the list of available models.
/// The availability depends on your API credentials and the current offerings from each provider.
/// Each provider's live results are merged with a curated static catalog, so
/// providers with a curated list (OpenAI, Anthropic, Gemini) still return their
/// well-known models even when no matching API key is configured. Providers without
/// a curated list (e.g. Ollama) are only returned when reachable.
///
/// # Returns
///
Expand Down
Loading