Updated internal documentation

This commit continues the process of documenting the core Starbeam
concepts for contributor consumption, using the package READMEs to
document the APIs of the individual packages.

This factoring is not designed for application developers, but it's
a perfect place to outline the structure of the concepts from the
perspective of someone trying to understand how the codebase is
structured.

.vscode/settings.json

@@ -30,13 +30,10 @@
   // trying to format the code.
   "editor.formatOnSave": false,
 
-  "eslint.codeActionsOnSave.mode": "all",
+  "eslint.codeActionsOnSave.mode": "problems",
   "eslint.enable": true,
   "eslint.lintTask.enable": true,
   "eslint.lintTask.options": "--cache -c ./.eslintrc.json --max-warnings=0 \"**/*.ts\" \"**/*.tsx\" \"**/*.json\"",
-  "eslint.onIgnoredFiles": "warn",
-  "eslint.problems.shortenToSingleLine": true,
-  "eslint.quiet": false,
   "eslint.validate": [
     "javascript",
     "javascriptreact",
@@ -81,6 +78,7 @@
   "inline-bookmarks.view.exclude.gitIgnore": true,
 
   "inline-bookmarks.view.showVisibleFilesOnly": false,
+  "markdown.validate.enabled": true,
   "npm-outdated.decorations": "fancy",
   // rewrap provides a quick keyboard shortcut (alt-q) to reformat comments to
   // a specific column width. It also automatically rewraps comments when you

README.md

@@ -8,17 +8,27 @@ Starbeam is a new kind of reactive library. It makes reactive programming simple
 
 </center>
 
+---
+
 [docs]: https://starbeamjs.com
 [discord]: https://discord.gg/HXq3PMmj8A
 
-It interoperates natively with React state management patterns, Svelte stores,
-the Vue composition API, and Ember's auto-tracking system.
+The core concepts of Starbeam are documented in the [docs] for users of Starbeam.
+
+The package READMEs have additional implementor-focused documentation:
 
-## The Big Picture
+- [@starbeam/shared]: Primitive Starbeam fundamentals. This package enables multiple
+  copies of Starbeam to interoperate, including across major versions of Starbeam.
+- [@starbeam/tags]: The core of Starbeam's demand-driven validation system.
+- [@starbeam/reactive]: The implementation of Starbeam's fundamental reactive
+  values (cells and formulas).
+- [@starbeam/collections]: Starbeam's reactive collections: reactive
+  implementations of JavaScript's built-in collections (object, array,
+  `Map`, `Set`, `WeakMap`, and `WeakSet`).
 
-- **Use normal JavaScript APIs and access patterns.** Build reactive data structures using reactive versions of JavaScript built-ins like objects, arrays, maps, and sets, and update them exactly as you would with normal JavaScript.
-- **No delays.** Updates to reactive values take effect immediately. Your reactive data and computations are always coherent, just like normal JavaScript.
-- **Works everywhere.** You don't need to learn a new framework to take advantage of Starbeam's cutting edge reactive system. You can render Starbeam data using renderers designed for your framework of choice, or even build your own renderer.
-- **TypeScript types are a first-class consideration.** Every part of Starbeam's API was designed from the ground up using TypeScript types with careful attention to the details. And if you're not a TypeScript user, you can still get the benefit in normal JavaScript code through inline suggestions, documentation and tab completion in your editor, no configuration needed.
+More READMEs are coming.
 
-Get started with Starbeam and learn more at the [docs website](https://starbeamjs.com).
+[@starbeam/shared]: ./packages/universal/shared/README.md
+[@starbeam/tags]: ./packages/universal/tags/README.md
+[@starbeam/reactive]: ./packages/universal/reactive/README.md
+[@starbeam/collections]: ./packages/universal/collections/README.md

packages/universal/collections/README.md

@@ -0,0 +1,120 @@
+# Starbeam Collections
+
+Starbeam collections are reactive implementations of JavaScript built-in
+collections.
+
+- [Object]
+- [Array]
+- [Map]
+- [Set]
+- [WeakMap]
+- [WeakSet]
+
+Starbeam collections behave identically to the JavaScript built-ins that they
+implement.
+
+> 📢 Starbeam Collections do not support prototype mutation. This is the only known
+> divergence from the JavaScript APIs.
+
+Internally, Starbeam collections model JavaScript read operations in one of
+three ways:
+
+- key accesses, which check whether a key is a member of the collection.
+- value accesses, which access and return a value from the collection.
+- key iterations, which iterate over a collection's keys.
+- value iterations, which iterate over a collection's values.
+
+## Example
+
+For example, in the `Map` API:
+
+- `has(key)` is a **key access**.
+- `get(key)` is an **value access**.
+- `keys()` is a **key iteration**.
+- `values()` is a **value iteration**.
+- `entries()` is a **key and value iteration**.
+- iterating the map via `Symbol.iterator` or `forEach` is a **key and value
+  iteration**.
+
+Consider this formula:
+
+```ts
+const recipes = reactive.Map(["pie", "http://example.com/pie-recipe"]);
+const hasTastyFood = Formula(() => food.has("pie") || food.has("cookie"));
+```
+
+This formula makes two _key accesses_: `"pie"` and `"cookie"`.
+
+If the URL for `pie` is updated:
+
+```ts
+recipes.set("pie", "http://example.com/better-pie-recipe");
+```
+
+This formula updates the _value_ of `"pie"`, but the _key_ has not changed. The
+`hasTastyFood` formula **will not invalidate**.
+
+Categorizing operations this way supports the intuition that `hasTastyFood` has
+not changed by providing enough granularity to capture the user's intent.
+
+## Iteration
+
+If a formula iterates over a reactive collection, the formula will invalidate
+when the collection changes.
+
+Replacing an existing value will invalidate a _value iteration_ but not a _key
+iteration_. Adding a new entry or deleting an existing entry will invalidate
+both.
+
+Let's create a couple of new formulas in our recipe example:
+
+```ts
+const recipeCount = Formula(() => recipes.size);
+const uniqueRecipeURLs = Formula(
+  () => new Set([...recipes.values()].map((r) => r.url))
+);
+```
+
+The `recipeCount` formula only changes when the _key iteration_ is invalidated.
+
+So let's say we update the pie recipe again:
+
+```ts
+recipes.set("pie", "http://example.com/even-better-pie-recipe");
+```
+
+Since changing the value of an existing entry doesn't invalidate the _key
+iteration_, the `recipeCount` formula **will not invalidate**.
+
+However, the `uniqueRecipeURLs` formula **will invalidate** when the
+collection changes.
+
+Intuitively, this behaves just as we'd expect: changing the value of a recipe
+doesn't change the size of the recipes collection. But it _might_ change the
+number of unique URLs.
+
+## 📢 Invalidation
+
+Starbeam collections have predictable, granular invalidation rules. They
+cannot, however, avoid invalidating a formula just because the formula
+ultimately produces the same answer.
+
+In this example, it's possible to invalidate `uniqueRecipeURLs` by changing
+the value of a recipe URL from a URL that has other existing entries to a URL
+that also has existing entries.
+
+In this situation, computing the formula will ultimately determine that
+nothing has changed. However, Starbeam invalidation rules are based upon the
+invalidation of storage cells, and **never** rely upon comparing the value of
+a previous computation with the value of a new computation.
+
+In practice, this means that renderers may want to compare the new value to
+the old value in order to determine whether to do the work of updating the
+rendererd output.
+
+[Object]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object
+[Array]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array
+[Map]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Map
+[Set]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Set
+[WeakMap]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/WeakMap
+[WeakSet]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/WeakSet

packages/universal/interfaces/src/tag.ts

@@ -1,5 +1,3 @@
-import type { UNINITIALIZED } from "@starbeam/shared";
-
 import type { Description } from "./debug/description.js";
 import type { Timestamp } from "./timestamp.js";
 
@@ -46,7 +44,7 @@ export interface CellTag {
 export interface FormulaTag {
   readonly type: "formula";
   readonly description?: Description | undefined;
-  readonly dependencies: UNINITIALIZED | (() => readonly CellTag[]);
+  readonly dependencies: undefined | (() => readonly CellTag[]);
 }
 
 /**

packages/universal/reactive/README.md

@@ -1,7 +1,10 @@
 This package contains the implementations of the reactive primitives.
 
 Reactive primitives must be used with an implementation of `Runtime`, which
-basically means that they must be used together with `@reactive/runtime`.
+basically means that they must be used together with `@starbeam/runtime`.
+
+Higher-level packages, such as `@starbeam/universal`, `@starbeam/resource` and
+the renderers include `@starbeam/runtime` as a dependency.
 
 > The primitives themselves, and higher-level concepts built on the primitives are
 > agnostic to the runtime, primarily to clearly mark the runtime interface and
@@ -53,6 +56,8 @@ stored elsewhere. For example, reactive collections store their values in the
 JavaScript collections they represent, and use markers to represent each
 discrete piece of storage in the collection.
 
+> You can think of a marker as a kind of cell without a value.
+
 A marker has these fundamental operations:
 
 - `mark()`: mark the external storage as being dirty.
@@ -64,3 +69,76 @@ A marker has these fundamental operations:
 > `get` for each entry in the map. This means that if a formula used a `has`
 > check to check for a key's presence and it returns `true`, updating the value
 > of that key will not invalidate the formula.
+
+### Formula
+
+A formula is a function that computes a reactive value. A formula's dependencies
+is the set of cells that were accessed during its last computation.
+
+A formula is a [reactive value](#the-reactive-protocol). Whenever the formula
+is `read()`, it recomputes its value. It is also a function. Calling the formula
+has the same behavior as calling the formula's `read()` method.
+
+#### Cached Formula
+
+A cached formula behaves like a formula, but it only recomputes its value when
+one of its dependencies changes.
+
+## Formula vs. CachedFormula
+
+Both formulas and cached formulas are reactive values. You can render either one
+(see `@starbeam/runtime` for more information). In either case, when the formula
+is recomputed, its dependencies are updated, and the formula's renderers will
+receive readiness notifications when any of the new dependencies change.
+
+The difference is that a cached formula will only recompute when one of its
+dependencies changes.
+
+Normal formulas are suitable for **mixed-reactive environments**, where a
+Starbeam formula uses both Starbeam reactive values **and** a framework's native
+reactive values.
+
+For example, consider this situation when using the React renderer:
+
+```ts
+function Counter() {
+  const [reactCount, setReactCount] = useState(0);
+
+  const starbeamCount = useSetup(() => {
+    const count = Cell(0);
+
+    return {
+      increment: () => {
+        count.current++;
+      },
+      get count() {
+        return count.read();
+      },
+    };
+  });
+
+  return useReactive(() => {
+    <p>
+      React count: {reactCount}
+      <button onClick={() => setReactCount(reactCount + 1)}>Increment</button>
+    </p>;
+    <p>
+      Starbeam count: {starbeamCount.count}
+      <button onClick={starbeamCount.increment}>Increment</button>
+    </p>;
+  });
+}
+```
+
+Under the hood, `useReactive` uses a normal formula, which will result in an
+updated output whenever either `reactCount` or `starbeamCount.count` changes.
+
+- If `reactCount` changes, React will re-render the component, and the formula
+  will be recomputed.
+- If `starbeamCount.count` changes, the formula will be recomputed, and React
+  will re-render the component.
+
+In practice, this makes `Formula` a good default choice for mixed-reactive
+environments. You can always use `CachedFormula` if you are confident that your
+formula doesn't use any reactive values external to Starbeam to optimize your
+code further.

packages/universal/reactive/src/primitives/formula.ts

@@ -1,8 +1,13 @@
-import type { FormulaTag, Tag, TaggedReactive } from "@starbeam/interfaces";
+import type {
+  FormulaTag,
+  Tag,
+  TaggedReactive,
+  TagSnapshot,
+} from "@starbeam/interfaces";
 import { TAG } from "@starbeam/shared";
 import { createFormulaTag } from "@starbeam/tags";
 
-import { evaluate, getDebug, getRuntime } from "../runtime.js";
+import { getDebug, getRuntime } from "../runtime.js";
 import type { FormulaFn, SugaryPrimitiveOptions } from "./utils.js";
 import { toOptions, WrapFn } from "./utils.js";
 
@@ -39,3 +44,10 @@ export function Formula<T>(
     },
   });
 }
+
+function evaluate<T>(compute: () => T): { value: T; tags: TagSnapshot } {
+  const done = getRuntime().start();
+  const value = compute();
+  const tags = done();
+  return { value, tags };
+}

packages/universal/reactive/src/runtime.ts

@@ -1,4 +1,4 @@
-import type { DebugRuntime, Runtime, TagSnapshot } from "@starbeam/interfaces";
+import type { DebugRuntime, Runtime } from "@starbeam/interfaces";
 import { isPresent, verified } from "@starbeam/verify";
 
 export const CONTEXT = {
@@ -40,10 +40,3 @@ if (import.meta.env.DEV) {
     DEBUG = debug;
   };
 }
-
-export function evaluate<T>(compute: () => T): { value: T; tags: TagSnapshot } {
-  const done = getRuntime().start();
-  const value = compute();
-  const tags = done();
-  return { value, tags };
-}

packages/universal/runtime/README.md

@@ -1,10 +1,12 @@
 # Purpose
 
-`@starbeam/timeline` is part of Starbeam, a library for building and using reactive objects in any framework.
+**This document is outdated, but still contains relevant philosophical information.**
+
+`@starbeam/runtime` is part of Starbeam, a library for building and using reactive objects in any framework.
 
 ## Primitive
 
-`@starbeam/timeline` is stable, with the same [semver policy as Starbeam][starbeam semver policy].
+`@starbeam/runtime` is stable, with the same [semver policy as Starbeam][starbeam semver policy].
 
 That said, it is not intended to be used directly by application code. Rather, it is one of the core parts of the Starbeam composition story. You can use it to better understand how Starbeam works, or to build your own Starbeam libraries.
 
@@ -79,7 +81,7 @@ These steps allow you to implement _framework-agnostic_ [resources] that can cor
 
 ## Timeline
 
-The `Timeline` in `@starbeam/timeline` coordinates these phases.
+The `Timeline` in `@starbeam/runtime` coordinates these phases.
 
 It starts out in the _Actions_ phase, which allows free access to the _data universe_. As soon as a _data cell_ in the _data universe_ is mutated, the `Timeline` schedules a _Render_ phase using the configured scheduler. By default, this will schedule a _Render_ phase during a microtask checkpoint, which occurs asynchronously, but before the next paint.
 

packages/universal/runtime/src/timeline/subscriptions.ts

@@ -6,7 +6,6 @@ import type {
   NotifyReady,
   Tag,
 } from "@starbeam/interfaces";
-import { UNINITIALIZED } from "@starbeam/shared";
 import { getDependencies, getTag } from "@starbeam/tags";
 
 import type { Unsubscribe } from "../lifetime/object-lifetime.js";
@@ -87,8 +86,8 @@ export class Subscriptions {
 
 function isUninitialized(
   tag: Tag,
-): tag is FormulaTag & { dependencies: UNINITIALIZED } {
-  return tag.dependencies === UNINITIALIZED;
+): tag is FormulaTag & { dependencies: undefined } {
+  return tag.dependencies === undefined;
 }
 
 /**
@@ -100,8 +99,7 @@ function hasDependencies(
   tagged: Tag,
 ): tagged is Tag & { readonly dependencies: () => readonly CellTag[] } {
   const deps = getTag(tagged).dependencies;
-
-  return deps !== UNINITIALIZED && isPresentArray(deps());
+  return deps !== undefined && isPresentArray(deps());
 }
 
 interface Subscription {