Skip to content

Commit 7ea26d7

Browse files
ultmasterultmaster
authored
Docs, tests, bug fixes on <meta> (#93)
1 parent 3fb2d52 commit 7ea26d7

File tree

9 files changed

+642
-34
lines changed

9 files changed

+642
-34
lines changed

‎docs/language/meta.md‎

Lines changed: 261 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,261 @@
1+
# Meta
2+
3+
The `<meta>` element provides metadata and configuration for POML documents. It allows you to specify version requirements, disable/enable components, define response schemas, register tools, and set runtime parameters.
4+
5+
## Basic Usage
6+
7+
Meta elements are typically placed at the beginning of a POML document and don't produce any visible output. One POML file can have multiple `<meta>` elements at any position, but they should be used carefully to avoid conflicts.
8+
9+
```xml
10+
<poml>
11+
<meta minVersion="1.0.0" />
12+
<p>Your content here</p>
13+
</poml>
14+
```
15+
16+
### Meta Element Types
17+
18+
Meta elements fall into two categories based on whether they include a `type` attribute:
19+
20+
**Without type attribute** - Used for general document configuration:
21+
- Version control (`minVersion`, `maxVersion`)
22+
- Component management (`components`)
23+
24+
**With type attribute** - Used for specific functionalities:
25+
- `type="responseSchema"` - Defines structured output format for AI responses
26+
- `type="tool"` - Registers callable functions for AI models
27+
- `type="runtime"` - Sets language model execution parameters
28+
29+
## Response Schema
30+
31+
Response schemas define the expected structure of AI-generated responses, ensuring that language models return data in a predictable, parsable format. This transforms free-form text generation into structured data generation.
32+
33+
### JSON Schema Format
34+
35+
Use the `lang="json"` attribute to specify JSON Schema format:
36+
37+
```xml
38+
<meta type="responseSchema" lang="json">
39+
{
40+
"type": "object",
41+
"properties": {
42+
"name": { "type": "string" },
43+
"age": { "type": "number" }
44+
},
45+
"required": ["name"]
46+
}
47+
</meta>
48+
```
49+
50+
### Expression Format
51+
52+
Use the `lang="expr"` attribute (or omit it for auto-detection) to evaluate JavaScript expressions that return schemas:
53+
54+
```xml
55+
<meta type="responseSchema" lang="expr">
56+
z.object({
57+
name: z.string(),
58+
age: z.number().optional()
59+
})
60+
</meta>
61+
```
62+
63+
When `lang` is omitted, POML auto-detects the format:
64+
- If the content starts with `{`, it's treated as JSON
65+
- Otherwise, it's treated as an expression
66+
67+
### Expression Evaluation in Schemas
68+
69+
#### JSON Schema with Template Expressions
70+
71+
JSON schemas support template expressions using `{{ }}` syntax:
72+
73+
```xml
74+
<let name="maxAge" value="100" />
75+
<meta type="responseSchema" lang="json">
76+
{
77+
"type": "object",
78+
"properties": {
79+
"name": { "type": "string" },
80+
"age": {
81+
"type": "number",
82+
"minimum": 0,
83+
"maximum": {{ maxAge }}
84+
}
85+
}
86+
}
87+
</meta>
88+
```
89+
90+
#### Expression Format with JavaScript Evaluation
91+
92+
Expression schemas are evaluated as JavaScript code with access to context variables and the `z` (Zod) variable:
93+
94+
```xml
95+
<let name="fields" value='["name", "email", "age"]' />
96+
<meta type="responseSchema" lang="expr">
97+
z.object(
98+
Object.fromEntries(fields.map(f => [f, z.string()]))
99+
)
100+
</meta>
101+
```
102+
103+
The expression can return either:
104+
- A Zod schema object (detected by the presence of `_def` property)
105+
- A plain JavaScript object treated as JSON Schema
106+
107+
**Important limitations:**
108+
- Only one `responseSchema` meta element is allowed per document. Multiple response schemas will result in an error.
109+
- Response schemas cannot be used together with tool definitions in the same document. You must choose between structured responses or tool calling capabilities.
110+
111+
## Tool Registration
112+
113+
Tool registration enables AI models to interact with external functions during conversation. Tools are function definitions that tell the AI model what functions are available, what parameters they expect, and what they do.
114+
115+
**Important:** Tools and response schemas are mutually exclusive. You cannot use both `responseSchema` and `tool` meta elements in the same POML document.
116+
117+
### JSON Schema Format
118+
119+
```xml
120+
<meta type="tool" name="getWeather" description="Get weather information">
121+
{
122+
"type": "object",
123+
"properties": {
124+
"location": { "type": "string" },
125+
"unit": {
126+
"type": "string",
127+
"enum": ["celsius", "fahrenheit"]
128+
}
129+
},
130+
"required": ["location"]
131+
}
132+
</meta>
133+
```
134+
135+
### Expression Format
136+
137+
```xml
138+
<meta type="tool" name="calculate" description="Perform calculation" lang="expr">
139+
z.object({
140+
operation: z.enum(['add', 'subtract', 'multiply', 'divide']),
141+
a: z.number(),
142+
b: z.number()
143+
})
144+
</meta>
145+
```
146+
147+
### Expression Evaluation in Tool Schemas
148+
149+
Tool schemas support the same evaluation modes as response schemas:
150+
151+
#### JSON with Template Expressions
152+
153+
```xml
154+
<let name="maxValue" value="1000" />
155+
<meta type="tool" name="calculator" description="Calculate values" lang="json">
156+
{
157+
"type": "object",
158+
"properties": {
159+
"value": {
160+
"type": "number",
161+
"maximum": {{ maxValue }}
162+
}
163+
}
164+
}
165+
</meta>
166+
```
167+
168+
#### Expression Format
169+
170+
```xml
171+
<let name="supportedOperations" value='["add", "subtract", "multiply", "divide"]' />
172+
<meta type="tool" name="calculator" description="Perform mathematical operations" lang="expr">
173+
z.object({
174+
operation: z.enum(supportedOperations),
175+
a: z.number(),
176+
b: z.number()
177+
})
178+
</meta>
179+
```
180+
181+
In expression mode, the `z` variable is automatically available for constructing Zod schemas, and you have direct access to all context variables.
182+
183+
**Required attributes for tools:**
184+
- **name**: Tool identifier (required)
185+
- **description**: Tool description (optional but recommended)
186+
- **lang**: Schema language, either "json" or "expr" (optional, auto-detected based on content)
187+
188+
You can define multiple tools in a single document.
189+
190+
## Runtime Parameters
191+
192+
Runtime parameters configure the language model's behavior during execution. These parameters are automatically used in [VSCode's test command](../vscode/features.md) functionality, which is based on the [Vercel AI SDK](https://ai-sdk.dev/).
193+
194+
```xml
195+
<meta type="runtime"
196+
temperature="0.7"
197+
maxOutputTokens="1000"
198+
model="gpt-5"
199+
topP="0.9" />
200+
```
201+
202+
All attributes except `type` are passed as runtime parameters. Common parameters include:
203+
204+
- **temperature**: Controls randomness (0-2, typically 0.3-0.7 for balanced output)
205+
- **maxOutputTokens**: Maximum response length in tokens
206+
- **model**: Model identifier (e.g., "gpt-5", "claude-4-sonnet")
207+
- **topP**: Nucleus sampling threshold (0-1, typically 0.9-0.95)
208+
- **frequencyPenalty**: Reduces token repetition based on frequency (-2 to 2)
209+
- **presencePenalty**: Reduces repetition based on presence (-2 to 2)
210+
- **seed**: For deterministic outputs (integer value)
211+
212+
The full parameter list depends on whether you're using standard text generation or structured data generation:
213+
- [Text generation parameters](https://ai-sdk.dev/docs/ai-sdk-core/generating-text) - Standard text generation
214+
- [Structured data parameters](https://ai-sdk.dev/docs/ai-sdk-core/generating-structured-data) - When using response schemas
215+
216+
The [Vercel AI SDK](https://ai-sdk.dev/) automatically handles parameter validation and conversion for different model providers.
217+
218+
## Version Control
219+
220+
Version requirements ensure compatibility between documents and the POML runtime. This prevents runtime errors when documents require specific POML features.
221+
222+
```xml
223+
<meta minVersion="0.5.0" maxVersion="2.0.0" />
224+
```
225+
226+
- **minVersion**: Minimum required POML version. If the current version is lower, an error is thrown.
227+
- **maxVersion**: Maximum supported POML version. Documents may not work correctly with newer versions.
228+
229+
Version checking uses semantic versioning (MAJOR.MINOR.PATCH) and occurs during document parsing.
230+
231+
## Component Control
232+
233+
The `components` attribute dynamically enables or disables POML components within a document. This is useful for conditional content, feature flags, or restricting elements in specific contexts.
234+
235+
### Disabling Components
236+
237+
Prefix component names with `-` to disable them:
238+
239+
```xml
240+
<meta components="-table" />
241+
<!-- Now <table> elements will throw an error -->
242+
```
243+
244+
You can disable multiple components:
245+
246+
```xml
247+
<meta components="-table,-image" />
248+
```
249+
250+
### Re-enabling Components
251+
252+
Use `+` prefix to re-enable previously disabled components:
253+
254+
```xml
255+
<meta components="-table" />
256+
<!-- table is disabled -->
257+
<meta components="+table" />
258+
<!-- table is re-enabled -->
259+
```
260+
261+
Component aliases can be disabled independently of the main component name. For example, if a component has both a main name and aliases, you can disable just the alias while keeping the main component available.

‎docs/media/vscode-evaluate.png‎

220 KB
Loading

‎docs/vscode/features.md‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,51 @@ While editing a `.poml` file in VSCode:
5959

6060
This feature significantly improves the efficiency and accuracy of writing POML code.
6161

62+
### Expression Evaluation with CodeLens
63+
64+
![Expression Evaluation](../media/vscode-evaluate.png)
65+
66+
The POML extension provides CodeLens buttons that allow you to evaluate template variables directly in your editor. This powerful debugging feature helps you understand what values your expressions produce locally.
67+
68+
#### How to Use Expression Evaluation
69+
70+
1. **CodeLens Buttons**: When you open a `.poml` file, you'll see "▶️ Evaluate" buttons appearing above expressions and variables.
71+
2. **Click to Evaluate**: Click any "▶️ Evaluate" button to execute the expression and see its result.
72+
3. **View Output**: Go to View → Output in VS Code. Results are displayed in the **POML Language Server** output channel.
73+
74+
#### What Gets Evaluated
75+
76+
The CodeLens evaluation feature works with:
77+
78+
- **Template Expressions**: Any `{{ expression }}` in your POML content
79+
- **Variable Definitions**: `<let>` element value attributes
80+
- **Control Flow**: Expressions in `for` and `if` attributes
81+
- **Schema Expressions**: Expressions in meta elements with `lang="expr"`
82+
83+
#### Example
84+
85+
```xml
86+
<poml>
87+
<let name="items" value='["apple", "banana", "cherry"]' />
88+
<let name="count" value="items.length" />
89+
90+
<p>We have {{ count }} items: {{ items.join(', ') }}</p>
91+
92+
<meta type="responseSchema" lang="expr">
93+
z.object({
94+
total: z.number().max(count),
95+
items: z.array(z.enum(items))
96+
})
97+
</meta>
98+
</poml>
99+
```
100+
101+
In this example, you can evaluate:
102+
- The `items` array definition to see `["apple", "banana", "cherry"]`
103+
- The `count` calculation to see `3`
104+
- The template expressions to see `"3"` and `"apple, banana, cherry"`
105+
- The schema expression to see the generated Zod schema object
106+
62107
## Testing Prompts
63108

64109
![Testing Prompts](../media/vscode-test.png)

‎mkdocs.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,7 @@ nav:
6363
- Getting Started: index.md
6464
- Quick Start: language/quickstart.md
6565
- Write .poml Files: language/standalone.md
66+
- Meta: language/meta.md
6667
- References:
6768
- Components: language/components.md
6869
- Deep Dive:

‎packages/poml-vscode/command/testCommand.ts‎

Lines changed: 27 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -97,14 +97,34 @@ export class TestCommand implements Command {
9797

9898
// Check if language model settings are configured
9999
const setting = this.getLanguageModelSettings(uri);
100-
if (!setting || !setting.provider || !setting.model || !setting.apiKey || !setting.apiUrl) {
101-
vscode.window.showErrorMessage(
102-
'Language model settings are not fully configured. Please set your provider, model, API key, and endpoint in the extension settings before testing prompts.'
103-
);
104-
this.log('error', 'Prompt test aborted: LLM settings not configured.');
100+
if (!setting) {
101+
vscode.window.showErrorMessage('Language model settings are not configured. Please configure your language model settings first.');
102+
this.log('error', 'Prompt test aborted: Language model settings not found.');
103+
return;
104+
}
105+
106+
if (!setting.provider) {
107+
vscode.window.showErrorMessage('Language model provider is not configured. Please set your provider in the extension settings.');
108+
this.log('error', 'Prompt test aborted: setting.provider is not configured.');
109+
return;
110+
}
111+
112+
if (!setting.model) {
113+
vscode.window.showErrorMessage('Language model is not configured. Please set your model in the extension settings.');
114+
this.log('error', 'Prompt test aborted: setting.model is not configured.');
115+
return;
116+
}
117+
118+
if (!setting.apiKey) {
119+
vscode.window.showErrorMessage('API key is not configured. Please set your API key in the extension settings.');
120+
this.log('error', 'Prompt test aborted: setting.apiKey not configured.');
105121
return;
106122
}
107123

124+
if (!setting.apiUrl) {
125+
this.log('info', 'No API URL configured, using default for the provider.');
126+
}
127+
108128
this.log(
109129
'info',
110130
`Testing prompt with ${this.isChatting ? 'chat model' : 'text completion model'}: ${fileUrl}`
@@ -294,7 +314,8 @@ export class TestCommand implements Command {
294314
} else if (chunk.type === 'abort') {
295315
return `${newline}[Aborted]`;
296316
} else if (chunk.type === 'error') {
297-
return `${newline}[Error: ${chunk.error}]`;
317+
// errors will be thrown, so we don't need to handle them here
318+
return null;
298319
} else {
299320
return `${newline}[${chunk.type} chunk: ${JSON.stringify(chunk)}]`;
300321
}

0 commit comments

Comments
 (0)