Skip to content

Commit 43ab9a1

Browse files
ricticrictic
authored
Merge pull request #44 from rictic/on-complete-value
Add completeCallback option
2 parents 39f1974 + e339012 commit 43ab9a1

7 files changed

Lines changed: 438 additions & 76 deletions

File tree

‎README.md‎

Lines changed: 73 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,8 @@ jsonriver is small, fast, has no dependencies, and uses only standard features o
77
Usage:
88

99
```js
10-
// Richer example at examples/fetch.js
10+
// Richer example at examples/fetch.js and live demos at
11+
// https://rictic.github.io/jsonriver/
1112
import {parse} from 'jsonriver';
1213

1314
const response = await fetch(`https://jsonplaceholder.typicode.com/posts`);
@@ -55,16 +56,83 @@ The `parse` function also matches `JSON.parse`'s behavior for invalid input. If
5556
we have the entire value.
5657
3. Strings may be replaced with a longer string, with more characters (in
5758
the JavaScript sense) appended.
58-
4. Arrays are only modified by either appending new elements, or
59+
4. Arrays are modified only by appending new elements, or
5960
replacing/mutating the element currently at the end.
6061
5. Objects are only modified by either adding new properties, or
6162
replacing/mutating the most recently added property, (except in the case of
6263
repeated keys, see invariant 7).
6364
6. As a consequence of 1 and 5, we only add a property to an object once we
6465
have the entire key and enough of the value to know that value's type.
65-
7. If an object has the same key multiple times, later values take precedence
66-
over earlier ones, matching the behavior of JSON.parse. This may result in
67-
changing the type of a value, and mutating earlier keys in the object.
66+
7. If an object has the same key multiple times, later values take
67+
precedence over earlier ones, matching the behavior of JSON.parse. This
68+
may result in changing the type of a value, and setting earlier keys
69+
the object.
70+
71+
## Complete Values
72+
73+
The parse function can be passed an options argument as its second parameter. If the options object has a `completeCallback` function, that function will be called like `completeCallback(value, path)` each time the parser has finished with a value.
74+
75+
Formally, a value is complete when jsonriver will not mutate it again, nor
76+
replace it with a different value, except for the unusual case of a
77+
repeated key in an object (see invariant 7 in the parse() docs).
78+
79+
The calls that jsonriver makes to a `completeCallback` are deterministic,
80+
regardless of how the incoming JSON streams in.
81+
82+
For example, when parsing this JSON:
83+
84+
```json
85+
{"name": "Alex", "keys": [1, 20, 300]}
86+
```
87+
88+
`completeCallback` will be called six times, with the following values:
89+
90+
```js
91+
'Alex'
92+
1
93+
20
94+
300
95+
[1, 20, 300]
96+
{"name": "Alex", "keys": [1, 20, 300]}
97+
```
98+
99+
It is also given a `path`, describing where the newly complete value is in relation to the toplevel parsed value. So for the above example, the paths are:
100+
101+
```js
102+
['name'] // `'Alex'` is in the 'keys' property on a toplevel object
103+
['keys', 0] // `1` is at index 0 in the array on the 'keys' prop
104+
['keys', 1] // `20` is at index 1 on that array
105+
['keys', 2] // `300` is at 2
106+
['keys'] // the array is complete, and found on the 'keys' property
107+
[] // finally, the toplevel object is complete
108+
```
109+
110+
This information is constructed lazily, so that you only pay for it if you use it. As a result, `completeCallback` must call `path.segments()` synchronously.
111+
112+
### Completions Recipe
113+
114+
A simple and low overhead way to handle completion information is with a WeakMap:
115+
116+
```js
117+
const completed = new WeakMap();
118+
function markCompleted(value) {
119+
if (value && typeof value === 'object') {
120+
completed.set(value, true);
121+
}
122+
}
123+
function isComplete(value) {
124+
if (value && typeof value === 'object') {
125+
return completed.has(value);
126+
}
127+
}
128+
129+
const values = parse(stream, {completeCallback: markCompleted});
130+
for await (const value of values) {
131+
// the render function can use the isComplete function to check whether
132+
// an object or array is complete
133+
render(value, isComplete);
134+
}
135+
```
68136

69137
## See also
70138

‎package-lock.json‎

Lines changed: 2 additions & 2 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎package.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
{
22
"name": "jsonriver",
33
"type": "module",
4-
"version": "1.0.3",
4+
"version": "1.1.0-beta.0",
55
"description": "A JSON parser that produces increasingly complete versions of the parsed value.",
66
"main": "dist/index.js",
77
"module": "dist/index.js",

‎src/parse.ts‎

Lines changed: 161 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -35,23 +35,89 @@ import {
3535
*
3636
* The following invariants will also be maintained:
3737
*
38-
* 1. Future versions of a value will have the same type. i.e. we will never
39-
* yield a value as a string and then later replace it with an array.
40-
* 2. true, false, null, and numbers are atomic, we don't yield them until
41-
* we have the entire value.
42-
* 3. Strings may be replaced with a longer string, with more characters (in
43-
* the JavaScript sense) appended.
44-
* 4. Arrays are only modified by either appending new elements, or
45-
* replacing/mutating the element currently at the end.
46-
* 5. Objects are only modified by either adding new properties, or
47-
* replacing/mutating the most recently added property.
48-
* 6. As a consequence of 1 and 5, we only add a property to an object once we
49-
* have the entire key and enough of the value to know that value's type.
38+
* 1. Subsequent versions of a value will have the same type. i.e. we will
39+
* never yield a value as a string and then later replace it with an array
40+
* (unless the object has repeated keys, see invariant 7).
41+
* 2. true, false, null, and numbers are atomic, we don't yield them until
42+
* we have the entire value.
43+
* 3. Strings may be replaced with a longer string, with more characters (in
44+
* the JavaScript sense) appended.
45+
* 4. Arrays are modified only by appending new elements or
46+
* replacing/mutating the element currently at the end.
47+
* 5. Objects are only modified by either adding new properties, or
48+
* replacing/mutating the most recently added property, (except in the case
49+
* of repeated keys, see invariant 7).
50+
* 6. As a consequence of 1 and 5, we only add a property to an object once we
51+
* have the entire key and enough of the value to know that value's type.
52+
* 7. If an object has the same key multiple times, later values take
53+
* precedence over earlier ones, matching the behavior of JSON.parse. This
54+
* may result in changing the type of a value, and setting earlier keys
55+
* the object.
5056
*/
5157
export async function* parse(
5258
stream: AsyncIterable<string>,
59+
options?: Options,
5360
): AsyncIterableIterator<JsonValue> {
54-
yield* new Parser(stream);
61+
yield* new Parser(stream, options?.completeCallback);
62+
}
63+
64+
interface Options {
65+
/**
66+
* A callback that's called with each value once that value is complete. It
67+
* will also be given information about the path to each
68+
* completed value.
69+
*
70+
* The calls that jsonriver makes to a `completeCallback` are deterministic,
71+
* regardless of how the incoming JSON streams in.
72+
*
73+
* Formally, a value is complete when jsonriver will not mutate it again, nor
74+
* replace it with a different value, except for the unusual case of a
75+
* repeated key in an object (see invariant 7 in the parse() docs).
76+
*
77+
* For example, when parsing this JSON:
78+
* ```json
79+
* {"name": "Alex", "keys": [1, 20, 300]}
80+
* ```
81+
*
82+
* The complete callback will be called six times, with the following values:
83+
*
84+
* ```js
85+
* "Alex"
86+
* 1
87+
* 20
88+
* 300
89+
* [1, 20, 300]
90+
* {"name": "Alex", "keys": [1, 20, 300]}
91+
* ```
92+
*
93+
* And the path segments would be:
94+
*
95+
* ```js
96+
* ['name'] // the 'keys' property on a toplevel object
97+
* ['keys', 0] // the 0th item in the array on the 'keys' prop
98+
* ['keys', 1] // the 1st item on that array
99+
* ['keys', 2] // the 2nd
100+
* ['keys'] // the 'keys' property is now complete
101+
* [] // finally, the toplevel value is complete
102+
* ```
103+
*/
104+
completeCallback?: (value: JsonValue, path: Path) => void;
105+
}
106+
107+
/**
108+
* The path of a complete value inside the toplevel parsed value.
109+
*
110+
* Note that Path values may be reused between calls to the complete callback.
111+
*/
112+
interface Path {
113+
/**
114+
* Constructs an array of the path to the most recently completed value.
115+
*
116+
* This method should be called synchronously when the completeCallback is
117+
* called, as the segments array is created lazily on demand based on the
118+
* parser's internal state.
119+
*/
120+
segments(): Array<string | number>;
55121
}
56122

57123
export type JsonValue =
@@ -110,6 +176,44 @@ interface InObjectExpectingValueState {
110176
value: [key: string, object: JsonObject];
111177
}
112178

179+
const privateStateStackSymbol = Symbol('stateStack');
180+
class CompleteValueInfo implements Path {
181+
private readonly [privateStateStackSymbol]: readonly State[];
182+
constructor(actualStateStack: State[]) {
183+
this[privateStateStackSymbol] = actualStateStack;
184+
}
185+
186+
segments(): Array<string | number> {
187+
const result = [];
188+
for (let i = 0; i < this[privateStateStackSymbol].length; i++) {
189+
const state = this[privateStateStackSymbol][i]!;
190+
switch (state.type) {
191+
case StateEnum.InString:
192+
case StateEnum.Initial:
193+
throw new Error(
194+
`path.segments() was called with unexpected parser state. Called asynchronously?`,
195+
);
196+
case StateEnum.InObjectExpectingKey:
197+
if (state.value[0] !== undefined) {
198+
result.push(state.value[0]);
199+
}
200+
continue;
201+
case StateEnum.InArray:
202+
result.push(state.value.length - 1);
203+
continue;
204+
case StateEnum.InObjectExpectingValue:
205+
result.push(state.value[0]);
206+
continue;
207+
default: {
208+
const never: never = state;
209+
throw new Error(`Unexpected state: ${String(never)}`);
210+
}
211+
}
212+
}
213+
return result;
214+
}
215+
}
216+
113217
class Parser implements AsyncIterableIterator<JsonValue>, TokenHandler {
114218
private readonly stateStack: State[] = [
115219
{type: StateEnum.Initial, value: undefined},
@@ -118,8 +222,15 @@ class Parser implements AsyncIterableIterator<JsonValue>, TokenHandler {
118222
readonly tokenizer: Tokenizer;
119223
private finished = false;
120224
private progressed = false;
225+
private completeCallback: Options['completeCallback'];
226+
private readonly completeValueInfo: CompleteValueInfo;
121227

122-
constructor(textStream: AsyncIterable<string>) {
228+
constructor(
229+
textStream: AsyncIterable<string>,
230+
completeCallback?: Options['completeCallback'],
231+
) {
232+
this.completeCallback = completeCallback;
233+
this.completeValueInfo = new CompleteValueInfo(this.stateStack);
123234
this.tokenizer = tokenize(textStream, this);
124235
}
125236

@@ -215,7 +326,7 @@ class Parser implements AsyncIterableIterator<JsonValue>, TokenHandler {
215326
}
216327
state.value += value;
217328
const parentState = this.stateStack[this.stateStack.length - 2];
218-
this.updateStringParent(state.value, parentState);
329+
this.updateStringParent(state.value, parentState, false);
219330
}
220331

221332
handleStringEnd(): void {
@@ -227,7 +338,7 @@ class Parser implements AsyncIterableIterator<JsonValue>, TokenHandler {
227338
}
228339
this.stateStack.pop();
229340
const parentState = this.stateStack[this.stateStack.length - 1];
230-
this.updateStringParent(state.value, parentState);
341+
this.updateStringParent(state.value, parentState, true);
231342
}
232343

233344
handleArrayStart(): void {
@@ -242,6 +353,9 @@ class Parser implements AsyncIterableIterator<JsonValue>, TokenHandler {
242353
);
243354
}
244355
this.stateStack.pop();
356+
if (this.completeCallback !== undefined) {
357+
this.completeCallback(state.value, this.completeValueInfo);
358+
}
245359
}
246360

247361
handleObjectStart(): void {
@@ -254,6 +368,9 @@ class Parser implements AsyncIterableIterator<JsonValue>, TokenHandler {
254368
case StateEnum.InObjectExpectingKey:
255369
case StateEnum.InObjectExpectingValue:
256370
this.stateStack.pop();
371+
if (this.completeCallback !== undefined) {
372+
this.completeCallback(state.value[1], this.completeValueInfo);
373+
}
257374
break;
258375
default:
259376
throw new Error(
@@ -279,10 +396,22 @@ class Parser implements AsyncIterableIterator<JsonValue>, TokenHandler {
279396
case StateEnum.Initial:
280397
this.stateStack.pop();
281398
this.toplevelValue = this.progressValue(type, value);
399+
if (
400+
this.completeCallback !== undefined &&
401+
this.stateStack.length === 0
402+
) {
403+
this.completeCallback(this.toplevelValue, this.completeValueInfo);
404+
}
282405
break;
283406
case StateEnum.InArray: {
284407
const v = this.progressValue(type, value);
285408
state.value.push(v);
409+
if (
410+
this.completeCallback !== undefined &&
411+
this.stateStack[this.stateStack.length - 1] === state
412+
) {
413+
this.completeCallback(v, this.completeValueInfo);
414+
}
286415
break;
287416
}
288417
case StateEnum.InObjectExpectingValue: {
@@ -298,6 +427,12 @@ class Parser implements AsyncIterableIterator<JsonValue>, TokenHandler {
298427
}
299428
const v = this.progressValue(type, value);
300429
setObjectProperty(object, key, v);
430+
if (
431+
this.completeCallback !== undefined &&
432+
this.stateStack[this.stateStack.length - 1] === expectedState
433+
) {
434+
this.completeCallback(v, this.completeValueInfo);
435+
}
301436
break;
302437
}
303438
case StateEnum.InString:
@@ -314,17 +449,27 @@ class Parser implements AsyncIterableIterator<JsonValue>, TokenHandler {
314449
private updateStringParent(
315450
updated: string,
316451
parentState: State | undefined,
452+
isFinal: boolean,
317453
): void {
318454
switch (parentState?.type) {
319455
case undefined:
320456
this.toplevelValue = updated;
457+
if (isFinal && this.completeCallback !== undefined) {
458+
this.completeCallback(updated, this.completeValueInfo);
459+
}
321460
break;
322461
case StateEnum.InArray:
323462
parentState.value[parentState.value.length - 1] = updated;
463+
if (isFinal && this.completeCallback !== undefined) {
464+
this.completeCallback(updated, this.completeValueInfo);
465+
}
324466
break;
325467
case StateEnum.InObjectExpectingValue: {
326468
const [key, object] = parentState.value;
327469
setObjectProperty(object, key, updated);
470+
if (isFinal && this.completeCallback !== undefined) {
471+
this.completeCallback(updated, this.completeValueInfo);
472+
}
328473
if (this.stateStack[this.stateStack.length - 1] === parentState) {
329474
this.stateStack.pop();
330475
this.stateStack.push({

0 commit comments

Comments
 (0)