Skip to content

Commit 0834c2d

Browse files
HT154HT154
committed
Implement Pkl binary renderer and parser
1 parent 6a06ab7 commit 0834c2d

37 files changed

+1717
-654
lines changed

‎docs/modules/bindings-specification/pages/binary-encoding.adoc‎

Lines changed: 26 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,9 @@ For example, value `8` gets encoded as MessagePack `int8` format.
3636
== Non-primitives
3737

3838
All non-primitive values are encoded as MessagePack arrays.
39-
The first slot of the array designates the value's type. The remaining slots have fixed meanings depending on the type.
39+
The first slot of the array designates the value's type.
40+
The remaining slots have fixed meanings depending on the type.
41+
Additional slots may be added to types in future Pkl releases, so decoders should be designed to defensively discard values beyond the number of known slots for a type or provide meaningful error messages.
4042

4143
The array's length is the number of slots that are filled. For example, xref:{uri-stdlib-List}[List] is encoded as an MessagePack array with two elements.
4244

@@ -46,7 +48,7 @@ The array's length is the number of slots that are filled. For example, xref:{ur
4648
||code |type |description |type |description |type |description
4749

4850
|link:{uri-stdlib-Typed}[Typed], link:{uri-stdlib-Dynamic}[Dynamic]
49-
|`0x1`
51+
|`0x01`
5052
|link:{uri-messagepack-str}[str]
5153
|Fully qualified class name
5254
|link:{uri-messagepack-str}[str]
@@ -55,7 +57,7 @@ The array's length is the number of slots that are filled. For example, xref:{ur
5557
|Array of <<object-members,object members>>
5658

5759
|link:{uri-stdlib-Map}[Map]
58-
|`0x2`
60+
|`0x02`
5961
|link:{uri-messagepack-map}[map]
6062
|Map of `<value>` to `<value>`
6163
|
@@ -64,7 +66,7 @@ The array's length is the number of slots that are filled. For example, xref:{ur
6466
|
6567

6668
|link:{uri-stdlib-Mapping}[Mapping]
67-
|`0x3`
69+
|`0x03`
6870
|link:{uri-messagepack-map}[map]
6971
|Map of `<value>` to `<value>`
7072
|
@@ -73,7 +75,7 @@ The array's length is the number of slots that are filled. For example, xref:{ur
7375
|
7476

7577
|link:{uri-stdlib-List}[List]
76-
|`0x4`
78+
|`0x04`
7779
|link:{uri-messagepack-array}[array]
7880
|Array of `<value>`
7981
|
@@ -82,7 +84,7 @@ The array's length is the number of slots that are filled. For example, xref:{ur
8284
|
8385

8486
|link:{uri-stdlib-Listing}[Listing]
85-
|`0x5`
87+
|`0x05`
8688
|link:{uri-messagepack-array}[array]
8789
|Array of `<value>`
8890
|
@@ -91,7 +93,7 @@ The array's length is the number of slots that are filled. For example, xref:{ur
9193
|
9294

9395
|link:{uri-stdlib-Set}[Set]
94-
|`0x6`
96+
|`0x06`
9597
|link:{uri-messagepack-array}[array]
9698
|Array of `<value>`
9799
|
@@ -100,7 +102,7 @@ The array's length is the number of slots that are filled. For example, xref:{ur
100102
|
101103

102104
|link:{uri-stdlib-Duration}[Duration]
103-
|`0x7`
105+
|`0x07`
104106
|{uri-messagepack-float}[float64]
105107
|Duration value
106108
|link:{uri-messagepack-str}[str]
@@ -109,7 +111,7 @@ The array's length is the number of slots that are filled. For example, xref:{ur
109111
|
110112

111113
|link:{uri-stdlib-DataSize}[DataSize]
112-
|`0x8`
114+
|`0x08`
113115
|link:{uri-messagepack-float}[float64]
114116
|Value (float64)
115117
|link:{uri-messagepack-str}[str]
@@ -118,7 +120,7 @@ The array's length is the number of slots that are filled. For example, xref:{ur
118120
|
119121

120122
|link:{uri-stdlib-Pair}[Pair]
121-
|`0x9`
123+
|`0x09`
122124
|`<value>`
123125
|First value
124126
|`<value>`
@@ -127,7 +129,7 @@ The array's length is the number of slots that are filled. For example, xref:{ur
127129
|
128130

129131
|link:{uri-stdlib-IntSeq}[IntSeq]
130-
|`0xA`
132+
|`0x0A`
131133
|link:{uri-messagepack-int}[int]
132134
|Start
133135
|link:{uri-messagepack-int}[int]
@@ -136,7 +138,7 @@ The array's length is the number of slots that are filled. For example, xref:{ur
136138
|Step
137139

138140
|link:{uri-stdlib-Regex}[Regex]
139-
|`0xB`
141+
|`0x0B`
140142
|link:{uri-messagepack-str}[str]
141143
|Regex string representation
142144
|
@@ -145,25 +147,25 @@ The array's length is the number of slots that are filled. For example, xref:{ur
145147
|
146148

147149
|link:{uri-stdlib-Class}[Class]
148-
|`0xC`
149-
|
150-
|
151-
|
152-
|
150+
|`0x0C`
151+
|link:{uri-messagepack-str}[str]
152+
|Module URI
153+
|link:{uri-messagepack-str}[str]
154+
|Qualified name
153155
|
154156
|
155157

156158
|link:{uri-stdlib-TypeAlias}[TypeAlias]
157-
|`0xD`
158-
|
159-
|
160-
|
161-
|
159+
|`0x0D`
160+
|link:{uri-messagepack-str}[str]
161+
|Module URI
162+
|link:{uri-messagepack-str}[str]
163+
|Qualified name
162164
|
163165
|
164166

165167
|link:{uri-stdlib-Function}[Function]
166-
|`0xE`
168+
|`0x0E`
167169
|
168170
|
169171
|
@@ -172,7 +174,7 @@ The array's length is the number of slots that are filled. For example, xref:{ur
172174
|
173175

174176
|link:{uri-stdlib-Bytes}[Bytes]
175-
|`0xF`
177+
|`0x0F`
176178
|link:{uri-messagepack-bin}[bin]
177179
|Binary contents
178180
|
@@ -212,4 +214,3 @@ Like non-primitive values, object members are encoded as MessagePack arrays, whe
212214
|`<value>`
213215
|element value
214216
|===
215-

‎pkl-commons-test/gradle.lockfile‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,5 +32,6 @@ org.junit.platform:junit-platform-commons:1.13.4=apiDependenciesMetadata,compile
3232
org.junit.platform:junit-platform-engine:1.13.4=apiDependenciesMetadata,compileClasspath,implementationDependenciesMetadata,runtimeClasspath,testCompileClasspath,testImplementationDependenciesMetadata,testRuntimeClasspath
3333
org.junit.platform:junit-platform-launcher:1.13.4=testRuntimeClasspath
3434
org.junit:junit-bom:5.13.4=apiDependenciesMetadata,compileClasspath,implementationDependenciesMetadata,runtimeClasspath,testCompileClasspath,testImplementationDependenciesMetadata,testRuntimeClasspath
35+
org.msgpack:msgpack-core:0.9.8=compileClasspath
3536
org.opentest4j:opentest4j:1.3.0=apiDependenciesMetadata,compileClasspath,implementationDependenciesMetadata,runtimeClasspath,testCompileClasspath,testImplementationDependenciesMetadata,testRuntimeClasspath
3637
empty=annotationProcessor,compileOnlyDependenciesMetadata,intransitiveDependenciesMetadata,kotlinCompilerPluginClasspath,kotlinNativeCompilerPluginClasspath,kotlinScriptDefExtensions,sourcesJar,testAnnotationProcessor,testApiDependenciesMetadata,testCompileOnlyDependenciesMetadata,testIntransitiveDependenciesMetadata,testKotlinScriptDefExtensions

‎pkl-commons-test/pkl-commons-test.gradle.kts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,7 @@ dependencies {
2828
api(libs.junitParams)
2929
api(projects.pklCommons) // for convenience
3030
implementation(libs.assertj)
31+
implementation(libs.msgpack)
3132
}
3233

3334
/**

‎pkl-core/src/main/java/org/pkl/core/runtime/BaseModule.java‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -111,6 +111,10 @@ public static VmClass getRenderDirectiveClass() {
111111
return RenderDirectiveClass.instance;
112112
}
113113

114+
public static VmClass getBytesRenderDirectiveClass() {
115+
return BytesRenderDirectiveClass.instance;
116+
}
117+
114118
/**
115119
* Returns class pkl.base#Module. For the module class of pkl.base use {@code
116120
* getModule().getVmClass()}.
@@ -327,6 +331,10 @@ private static final class RenderDirectiveClass {
327331
static final VmClass instance = loadClass("RenderDirective");
328332
}
329333

334+
private static final class BytesRenderDirectiveClass {
335+
static final VmClass instance = loadClass("BytesRenderDirective");
336+
}
337+
330338
private static final class PairClass {
331339
static final VmClass instance = loadClass("Pair");
332340
}

‎pkl-core/src/main/java/org/pkl/core/runtime/EncodingModule.java‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
/*
2+
* Copyright © 2025 Apple Inc. and the Pkl project authors. All rights reserved.
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* https://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing, software
11+
* distributed under the License is distributed on an "AS IS" BASIS,
12+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
* See the License for the specific language governing permissions and
14+
* limitations under the License.
15+
*/
16+
package org.pkl.core.runtime;
17+
18+
import java.net.URI;
19+
20+
public class EncodingModule extends StdLibModule {
21+
private static final VmTyped instance = VmUtils.createEmptyModule();
22+
23+
static {
24+
loadModule(URI.create("pkl:encoding"), instance);
25+
}
26+
27+
public static VmTyped getModule() {
28+
return instance;
29+
}
30+
}

‎pkl-core/src/main/java/org/pkl/core/runtime/ModuleCache.java‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
/*
2-
* Copyright © 2024 Apple Inc. and the Pkl project authors. All rights reserved.
2+
* Copyright © 2024-2025 Apple Inc. and the Pkl project authors. All rights reserved.
33
*
44
* Licensed under the Apache License, Version 2.0 (the "License");
55
* you may not use this file except in compliance with the License.
@@ -92,6 +92,8 @@ public synchronized VmTyped getOrLoad(
9292
return BaseModule.getModule();
9393
case "Benchmark":
9494
return BenchmarkModule.getModule();
95+
case "encoding":
96+
return EncodingModule.getModule();
9597
case "jsonnet":
9698
return JsonnetModule.getModule();
9799
case "math":

0 commit comments

Comments
 (0)