Skip to content

Commit 16ce6f0

Browse files
authored
Merge pull request #294 from ueberdosis:feature/schema-awareness-cherrypick
Merge pull request #288 from ueberdosis/feature/schema-awareness
2 parents 39b63fe + 8a76085 commit 16ce6f0

File tree

11 files changed

+308
-28
lines changed

11 files changed

+308
-28
lines changed
Lines changed: 207 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,207 @@
1+
---
2+
title: Configure custom nodes and marks
3+
meta:
4+
title: Custom nodes/marks | Tiptap Content AI
5+
description: Learn how to configure the AI Agent extension so that it can generate and understand custom nodes and marks.
6+
category: Content AI
7+
---
8+
9+
import { CodeDemo } from '@/components/CodeDemo'
10+
11+
The AI Agent extension understands the content of the document and the possible elements (nodes and marks) that can be in it.
12+
13+
For example, the AI Agent extension detects the [Table](/editor/extensions/nodes/table) extension and knows how to generate tables. If the Table extension is not enabled and the user asks it to generate a table, the AI Agent will respond that the document doesn't support this type of element.
14+
15+
This is automatically done for all built-in Tiptap extensions. However, if you have an editor with [custom nodes and marks](/editor/extensions/custom-extensions), you need to configure the AI Agent extension so that it can generate and understand them.
16+
17+
<CodeDemo
18+
isPro
19+
isLarge
20+
path="/Extensions/AiAgentCustomElements"
21+
src="https://develop--tiptap-pro.netlify.app/preview/Extensions/AiAgentCustomElements"
22+
/>
23+
24+
## Configure custom nodes
25+
26+
To configure custom nodes for the AI Agent, you need to follow these steps:
27+
28+
1. Define your custom Tiptap node extension
29+
2. Configure the AI Agent with schema awareness information
30+
31+
Let's walk through each step with a practical example.
32+
33+
### Step 1: Define a custom Tiptap node extension
34+
35+
First, create your custom node extension. Here's an example of a custom "Alert" node:
36+
37+
```ts
38+
import { Node, mergeAttributes } from '@tiptap/core'
39+
40+
export const AlertNode = Node.create({
41+
name: 'alert',
42+
43+
addOptions() {
44+
return {
45+
HTMLAttributes: {},
46+
}
47+
},
48+
49+
addAttributes() {
50+
return {
51+
type: {
52+
default: 'info',
53+
parseHTML: (element) => element.getAttribute('data-type'),
54+
renderHTML: (attributes) => {
55+
if (!attributes.type) {
56+
return {}
57+
}
58+
return {
59+
'data-type': attributes.type,
60+
}
61+
},
62+
},
63+
}
64+
},
65+
66+
parseHTML() {
67+
return [
68+
{
69+
tag: 'div[data-alert]',
70+
},
71+
]
72+
},
73+
74+
renderHTML({ HTMLAttributes }) {
75+
return [
76+
'div',
77+
mergeAttributes(this.options.HTMLAttributes, HTMLAttributes, {
78+
'data-alert': '',
79+
}),
80+
0,
81+
]
82+
},
83+
})
84+
```
85+
86+
Add the extension to your editor:
87+
88+
```ts
89+
import { Editor } from '@tiptap/core'
90+
import { AlertNode } from './AlertNode'
91+
92+
const editor = new Editor({
93+
extensions: [
94+
// ... other extensions
95+
AlertNode,
96+
],
97+
})
98+
```
99+
100+
### Step 2: Configure the AI Agent
101+
102+
Now configure the AI Agent extension with schema awareness information about your custom node:
103+
104+
```ts
105+
const provider = new AiAgentProvider({
106+
schemaAwarenessCustomElements: [
107+
{
108+
extensionName: 'alert',
109+
tag: 'div[data-alert]',
110+
name: 'Alert Box',
111+
description:
112+
'A highlighted box used to display important information, warnings, or tips to the user',
113+
// Describe the HTML attributes of the node as it is rendered in HTML
114+
attributes: [
115+
{
116+
name: 'data-alert',
117+
// Specify the "value" property if the attribute always has that value
118+
value: '',
119+
description: 'Indicates that this is an alert box',
120+
},
121+
{
122+
name: 'data-type',
123+
description: 'The type of alert: info, warning, error, or success',
124+
},
125+
],
126+
},
127+
],
128+
})
129+
```
130+
131+
## Multiple custom elements
132+
133+
You can configure multiple custom nodes and marks at once. Here's an example with both an Alert node and a custom highlight marks:
134+
135+
```ts
136+
const provider = new AiAgentProvider({
137+
schemaAwarenessCustomElements: [
138+
{
139+
extensionName: 'alert',
140+
tag: 'div[data-alert]',
141+
name: 'Alert Box',
142+
description: 'A highlighted box used to display important information, warnings, or tips',
143+
attributes: [
144+
{
145+
name: 'data-alert',
146+
value: '',
147+
description: 'Indicates that this is an alert box',
148+
}
149+
{
150+
name: 'data-type',
151+
description: 'The type of alert: info, warning, error, or success',
152+
},
153+
],
154+
},
155+
// Custom marks are also configured in the same way
156+
{
157+
extensionName: 'customHighlight',
158+
tag: 'span',
159+
name: 'Custom Highlight',
160+
description: 'Highlights text with a special background',
161+
attributes: [
162+
{
163+
name: 'data-custom-highlight',
164+
value: '',
165+
description: 'Indicates that this is a custom highlight',
166+
}
167+
],
168+
},
169+
],
170+
})
171+
```
172+
173+
## Best practices
174+
175+
When configuring custom nodes and marks for the AI Agent extension:
176+
177+
1. **Use descriptive names**: Choose clear, descriptive names that help the AI model understand what the element represents.
178+
179+
2. **Provide detailed descriptions**: Include comprehensive descriptions that explain both what the element is and how it's displayed or used.
180+
181+
3. **Document all attributes**: List all possible HTML attributes with their purposes and expected values.
182+
183+
4. **Use consistent naming**: Match the `extensionName` with your actual Tiptap extension name.
184+
185+
5. **Specify HTML structure**: Ensure `tag` and `atributes` match exactly how your extension renders HTML.
186+
187+
## API Reference
188+
189+
The `schemaAwarenessCustomElements` option accepts an array of `SchemaAwarenessItem` objects with the following properties:
190+
191+
| Property | Type | Required | Description |
192+
| --------------- | -------------------------------- | -------- | --------------------------------------------------------- |
193+
| `extensionName` | `string` | Yes | The name of the Tiptap extension (must match exactly) |
194+
| `tag` | `string` | Yes | The HTML tag or selector that represents this element |
195+
| `name` | `string` | Yes | A human-readable name for the element |
196+
| `description` | `string \| null` | No | Explanation of what the element is and how it's displayed |
197+
| `attributes` | `SchemaAwarenessItemAttribute[]` | No | Array of possible HTML attributes for this element |
198+
199+
### SchemaAwarenessItemAttribute
200+
201+
Each attribute object in the `attributes` array has the following properties:
202+
203+
| Property | Type | Required | Description |
204+
| ------------- | ---------------- | -------- | ---------------------------------------------------------- |
205+
| `name` | `string` | Yes | The name of the attribute in the HTML code |
206+
| `value` | `string` | No | If specified, the attribute always has this exact value |
207+
| `description` | `string \| null` | No | Explanation of the attribute's purpose and expected values |

src/content/content-ai/capabilities/agent/configure/options.mdx

Lines changed: 33 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -229,25 +229,38 @@ const provider = new AiAgentProvider({
229229
})
230230
```
231231

232+
## Custom nodes
233+
234+
Configure the AI Agent extension so that it can [generate and understand custom nodes](/content-ai/capabilities/agent/configure/custom-nodes).
235+
236+
```ts
237+
const provider = new AiAgentProvider({
238+
schemaAwarenessCustomElements: [
239+
/* Custom node configuration */
240+
],
241+
})
242+
```
243+
232244
## Key configuration options
233245

234-
| Option | Type | Default | Description |
235-
| ----------------------- | ----------------------------------- | -------------------------- | --------------------------------------------------------------------- |
236-
| `appId` | `string` | `""` | Your Tiptap Content AI app ID |
237-
| `token` | `string` | `""` | JWT token for authentication |
238-
| `baseUrl` | `string` | `""` | Base URL of the AI service API |
239-
| `modelName` | `AiAgentModelName` | `"gpt-4.1"` | The OpenAI model to use (gpt-4.1 and gpt-4o recommended) |
240-
| `autoAccept` | `"always" \| "never" \| "onlyRead"` | `"onlyRead"` | Controls automatic acceptance of AI changes |
241-
| `autoSaveCheckpoints` | `boolean` | `false` | Automatically save checkpoints when user sends a message |
242-
| `chunkSize` | `number` | `1000` | Size of document chunks when reading (in characters) |
243-
| `chunkHtml` | `Function` | `defaultChunkHtmlFunction` | Customizes how the document is split into chunks |
244-
| `useAiChangesExtension` | `boolean` | `true` | Whether to use the AI Changes extension |
245-
| `initialChatMessages` | `ChatMessage[]` | `[]` | Initial chat messages to populate the conversation |
246-
| `resolver` | `Function` | `defaultAiAgentResolver` | Function to resolve AI Agent requests with a custom backend |
247-
| `toolHandlers` | `AiAgentToolCallHandler[]` | `toolHandlersStarterKit()` | Handlers for custom tools |
248-
| `onStateChange` | `Function` | `undefined` | Called when the state of the AI Agent changes |
249-
| `onLoadingError` | `Function` | `undefined` | Called when there's an error loading the AI Agent |
250-
| `onBeforeToolCall` | `Function` | `undefined` | Called before a tool call is executed |
251-
| `onAfterToolCall` | `Function` | `undefined` | Called after a tool call is executed |
252-
| `onStopRunning` | `Function` | `undefined` | Called when the AI Agent stops running |
253-
| `systemPrompt` | `string` | `undefined` | Custom system prompt for the AI Agent when using it with Tiptap Cloud |
246+
| Option | Type | Default | Description |
247+
| ------------------------------- | ----------------------------------- | -------------------------- | --------------------------------------------------------------------------------- |
248+
| `appId` | `string` | `""` | Your Tiptap Content AI app ID |
249+
| `token` | `string` | `""` | JWT token for authentication |
250+
| `baseUrl` | `string` | `""` | Base URL of the AI service API |
251+
| `modelName` | `AiAgentModelName` | `"gpt-4.1"` | The OpenAI model to use (gpt-4.1 and gpt-4o recommended) |
252+
| `autoAccept` | `"always" \| "never" \| "onlyRead"` | `"onlyRead"` | Controls automatic acceptance of AI changes |
253+
| `autoSaveCheckpoints` | `boolean` | `false` | Automatically save checkpoints when user sends a message |
254+
| `chunkSize` | `number` | `1000` | Size of document chunks when reading (in characters) |
255+
| `chunkHtml` | `Function` | `defaultChunkHtmlFunction` | Customizes how the document is split into chunks |
256+
| `useAiChangesExtension` | `boolean` | `true` | Whether to use the AI Changes extension |
257+
| `initialChatMessages` | `ChatMessage[]` | `[]` | Initial chat messages to populate the conversation |
258+
| `resolver` | `Function` | `defaultAiAgentResolver` | Function to resolve AI Agent requests with custom backend |
259+
| `toolHandlers` | `AiAgentToolCallHandler[]` | `toolHandlersStarterKit()` | Handlers for custom tools |
260+
| `onStateChange` | `Function` | `undefined` | Called when the state of the AI Agent changes |
261+
| `onLoadingError` | `Function` | `undefined` | Called when there's an error loading the AI Agent |
262+
| `onBeforeToolCall` | `Function` | `undefined` | Called before a tool call is executed |
263+
| `onAfterToolCall` | `Function` | `undefined` | Called after a tool call is executed |
264+
| `onStopRunning` | `Function` | `undefined` | Called when the AI Agent stops running |
265+
| `systemPrompt` | `string` | `undefined` | Custom system prompt for the AI Agent when using it with Tiptap Cloud |
266+
| `schemaAwarenessCustomElements` | `SchemaAwarenessItem[]` | `[]` | Information for the AI model about the custom nodes that the document can contain |

src/content/content-ai/capabilities/agent/custom-llms/get-started/anthropic-messages.mdx

Lines changed: 13 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -49,11 +49,11 @@ import AiAgent, { AiAgentProvider } from '@tiptap-pro/extension-ai-agent'
4949

5050
const provider = new AiAgentProvider({
5151
// The `chatMessages` property contains the chat messages of the conversation
52-
resolver: async ({ chatMessages }) => {
52+
resolver: async ({ chatMessages, schemaAwarenessData }) => {
5353
// Call the API endpoint of your backend
5454
const response = await fetch('/api-endpoint', {
5555
method: 'POST',
56-
body: JSON.stringify({ chatMessages }),
56+
body: JSON.stringify({ chatMessages, schemaAwarenessData }),
5757
})
5858
return await response.json()
5959
},
@@ -70,13 +70,21 @@ First, install the AI Agent and Anthropic server libraries.
7070
npm install @tiptap-pro/extension-ai-agent-server @anthropic-ai/sdk
7171
```
7272

73+
Get the chat messages and schema awareness data from the request parameters.
74+
75+
```ts
76+
// Code inside your API endpoint. Code depends on your backend framework
77+
const { chatMessages, schemaAwarenessData } = request
78+
```
79+
7380
Then, inside your API endpoint, create an `AiAgentToolkit` instance. It lets you configure the tools that will be available to the AI model.
7481

7582
```ts
7683
import { AiAgentToolkit, anthropicMessagesAdapter } from '@tiptap-pro/extension-ai-agent-server'
7784

7885
const toolkit = new AiAgentToolkit({
7986
adapter: anthropicMessagesAdapter,
87+
schemaAwarenessData,
8088
})
8189
```
8290

@@ -105,8 +113,11 @@ import {
105113
} from '@tiptap-pro/extension-ai-agent-server'
106114
import Anthropic from '@anthropic-ai/sdk'
107115

116+
const { chatMessages, schemaAwarenessData } = request
117+
108118
const toolkit = new AiAgentToolkit({
109119
adapter: anthropicMessagesAdapter,
120+
schemaAwarenessData,
110121
})
111122

112123
const formatter = new ChatMessagesFormatter({

src/content/content-ai/capabilities/agent/custom-llms/get-started/openai-chat-completions.mdx

Lines changed: 13 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -49,11 +49,11 @@ import AiAgent, { AiAgentProvider } from '@tiptap-pro/extension-ai-agent'
4949

5050
const provider = new AiAgentProvider({
5151
// The `chatMessages` property contains the chat messages of the conversation
52-
resolver: async ({ chatMessages }) => {
52+
resolver: async ({ chatMessages, schemaAwarenessData }) => {
5353
// Call the API endpoint of your backend
5454
const response = await fetch('/api-endpoint', {
5555
method: 'POST',
56-
body: JSON.stringify({ chatMessages }),
56+
body: JSON.stringify({ chatMessages, schemaAwarenessData }),
5757
})
5858
return await response.json()
5959
},
@@ -70,13 +70,21 @@ First, install the AI Agent and OpenAI server libraries.
7070
npm install @tiptap-pro/extension-ai-agent-server openai
7171
```
7272

73+
Get the chat messages and schema awareness data from the request parameters.
74+
75+
```ts
76+
// Code inside your API endpoint. Code depends on your backend framework
77+
const { chatMessages, schemaAwarenessData } = request
78+
```
79+
7380
Then, inside your API endpoint, create an `AiAgentToolkit` instance. It lets you configure the tools that will be available to the AI model.
7481

7582
```ts
7683
import { AiAgentToolkit, openaiChatCompletionsAdapter } from '@tiptap-pro/extension-ai-agent-server'
7784

7885
const toolkit = new AiAgentToolkit({
7986
adapter: openaiChatCompletionsAdapter,
87+
schemaAwarenessData,
8088
})
8189
```
8290

@@ -105,8 +113,11 @@ import {
105113
} from '@tiptap-pro/extension-ai-agent-server'
106114
import OpenAI from 'openai'
107115

116+
const { chatMessages, schemaAwarenessData } = request
117+
108118
const toolkit = new AiAgentToolkit({
109119
adapter: openaiChatCompletionsAdapter,
120+
schemaAwarenessData,
110121
})
111122

112123
const formatter = new ChatMessagesFormatter({

0 commit comments

Comments
 (0)