Allow multiple triggers for a node (#49)

.github/workflows/publish-package.yml

@@ -7,9 +7,15 @@ on:
 jobs:
   pypi:
     runs-on: ubuntu-latest
+    environment:
+      name: pypi
+      url: https://pypi.org/p/${{ github.event.repository.name }}
+    permissions:
+      contents: read  # add default back in
+      id-token: write
     steps:
     - name: Check out repository
-      uses: actions/[email protected].7
+      uses: actions/[email protected].5
     - name: Set up Python 3.x
       uses: actions/[email protected]
       with:
@@ -18,6 +24,5 @@ jobs:
       uses: snok/[email protected]
     - name: Build the package
       run: poetry build -vvv
-    - name: Publish to PyPI
-      run: poetry publish --username __token__ --password ${{ secrets.pypi_token }}
-      working-directory: ${{ github.workspace }}
+    - name: Publish package distributions to PyPI
+      uses: pypa/[email protected]

.pre-commit-config.yaml

@@ -1,5 +1,4 @@
 repos:
-
   - repo: https://github.com/pre-commit/pre-commit-hooks
     rev: v4.6.0
     hooks:

README.md

@@ -10,3 +10,8 @@
 
 [![GitHub issues](https://img.shields.io/github/issues/JoshKarpel/synthesize)](https://github.com/JoshKarpel/synthesize/issues)
 [![GitHub pull requests](https://img.shields.io/github/issues-pr/JoshKarpel/synthesize)](https://github.com/JoshKarpel/synthesize/pulls)
+
+Synthesize is a tool for managing long-lived development workflows that involve multiple tools executing concurrently,
+each of which might have bespoke conditions around when and how it needs to be run or re-run.
+
+See [the documentation](https://www.synth.how) for more information.

docs/changelog.md

@@ -2,7 +2,7 @@
 
 ## `0.0.3`
 
-*Unreleased*
+Released `2024-07-07`
 
 ### Added
 
@@ -25,6 +25,8 @@
   and flows (graphs of targets and triggers)."
 - [#41](https://github.com/JoshKarpel/synthesize/pull/41)
   Execution duration is printed in the completion message.
+- [#49](https://github.com/JoshKarpel/synthesize/pull/49)
+  Flow nodes can now have multiple triggers.
 
 ### Fixed
 

docs/config.md

@@ -0,0 +1,3 @@
+# Config
+
+@schema(synthesize.config, Config)

docs/examples/after.yaml

@@ -9,16 +9,16 @@ flows:
         target: sleep-and-echo
       D:
         target: sleep-and-echo
-        trigger:
-          after: ["A", "B"]
+        triggers:
+        - after: [A, B]
       E:
         target: sleep-and-echo
-        trigger:
-          after: ["C"]
+        triggers:
+        - after: [C]
       F:
         target: sleep-and-echo
-        trigger:
-          after: ["D", "E"]
+        triggers:
+        - after: [D, E]
 
 targets:
   sleep-and-echo:

docs/examples/restart-and-watch.yaml

@@ -0,0 +1,9 @@
+flows:
+  default:
+    nodes:
+      docs:
+        target:
+          commands: mkdocs serve --strict
+        triggers:
+        - delay: 1
+        - watch: ["docs/hooks/"]

docs/examples/restart.yaml

@@ -3,14 +3,12 @@ flows:
     nodes:
       A:
         target: sleep-and-echo
-        trigger:
-          type: restart
-          delay: 3
+        triggers:
+        - delay: 3
       B:
         target: sleep-and-echo
-        trigger:
-          type: restart
-          delay: 1
+        triggers:
+        - delay: 1
 
 targets:
   sleep-and-echo:

docs/examples/watch.yaml

@@ -3,14 +3,12 @@ flows:
     nodes:
       A:
         target: sleep-and-echo
-        trigger:
-          type: watch
-          paths: ["synthesize/", "tests/"]
+        triggers:
+        - watch: ["synthesize/", "tests/"]
       B:
         target: sleep-and-echo
-        trigger:
-          type: watch
-          paths: [ "docs/" ]
+        triggers:
+        - watch: [ "docs/" ]
 
 targets:
   sleep-and-echo:

docs/flows.md

@@ -0,0 +1,5 @@
+# Flows
+
+@schema(synthesize.config, Flow)
+
+@schema(synthesize.config, Node)

docs/hooks/schemas.py

@@ -0,0 +1,129 @@
+import importlib
+import logging
+import re
+from collections.abc import Iterator, Mapping
+
+from mkdocs.config.defaults import MkDocsConfig
+from mkdocs.structure.files import Files
+from mkdocs.structure.pages import Page
+from openapi_pydantic import DataType, Reference, Schema
+
+logger = logging.getLogger("mkdocs")
+
+INDENT = " " * 4
+
+
+def on_page_markdown(
+    markdown: str,
+    page: Page,
+    config: MkDocsConfig,
+    files: Files,
+) -> str:
+    lines = []
+    for line in markdown.splitlines():
+        if match := re.match(r"@schema\(([\w\.]+)\s*\,\s*(\w*)\)", line):
+            mod = importlib.import_module(match.group(1))
+            importlib.reload(mod)
+            model = getattr(mod, match.group(2))
+            # lines.append(f"```json\n{json.dumps(model.model_json_schema(), indent=2)}\n```")
+            schema_raw = model.model_json_schema()
+            schema = Schema.model_validate(schema_raw)
+            defs = {k: Schema.model_validate(v) for k, v in schema_raw.get("$defs", {}).items()}
+            lines.append("")
+            lines.extend(schema_lines(schema, None, defs))
+            lines.append("")
+        else:
+            lines.append(line)
+
+    return "\n".join(lines)
+
+
+def indent(lines: Iterator[str]) -> Iterator[str]:
+    return (INDENT + l for l in lines)
+
+
+sep = "○"
+
+
+def schema_lines(
+    schema_or_ref: Schema | Reference, key: str | None, defs: Mapping[str, Schema]
+) -> Iterator[str]:
+    schema = ref_to_schema(schema_or_ref, defs)
+
+    dt = italic(display_type(schema, defs))
+
+    st = schema.title
+    assert st is not None
+
+    if schema.type in {DataType.STRING, DataType.NUMBER, DataType.BOOLEAN}:
+        t = mono(st.lower()) if st else ""
+        default = f" (Default: {mono(repr(schema.default))}) " if not schema.required else " "
+        yield f"- {t} {dt} {default} {sep if schema.description else ''} {schema.description}"
+    elif schema.type is DataType.ARRAY:
+        t = mono(st.lower()) if st else ""
+        yield f"- {t} {dt} {sep if schema.description else ''} {schema.description}"
+    elif schema.type is DataType.OBJECT:
+        default = (
+            f" (Default: {mono(repr(schema.default))}) " if key and not schema.required else " "
+        )
+        yield f"- {key or st.title()} {dt} {default} {sep if schema.description else ''} {schema.description or ''}"
+        if not schema.properties:
+            return
+        for k, prop in schema.properties.items():
+            yield from indent(schema_lines(prop, mono(k), defs))
+    elif schema.type is None:
+        if schema.anyOf:
+            yield f"- {mono(st.lower())} {dt} {sep if schema.description else ''} {schema.description}"
+        else:
+            raise NotImplementedError(
+                f"Type {schema.type} not implemented. Appeared in the schema for {st}: {schema!r}."
+            )
+    else:
+        raise NotImplementedError(
+            f"Type {schema.type} not implemented. Appeared in the schema for {st}: {schema!r}."
+        )
+
+
+def ref_to_schema(schema_or_ref: Schema | Reference, defs: Mapping[str, Schema]) -> Schema:
+    if isinstance(schema_or_ref, Reference):
+        try:
+            return defs[schema_or_ref.ref.removeprefix("#/$defs/")]
+        except KeyError:
+            logger.error(f"Could not find reference {schema_or_ref.ref!r} in {defs.keys()!r}")
+            raise
+    else:
+        return schema_or_ref
+
+
+def display_type(schema: Schema | Reference, defs: Mapping[str, Schema]) -> str:
+    schema = ref_to_schema(schema, defs)
+
+    st = schema.type
+
+    if isinstance(st, DataType) and st in {
+        DataType.STRING,
+        DataType.NUMBER,
+        DataType.BOOLEAN,
+        DataType.OBJECT,
+    }:
+        return str(st.value)
+    elif st is DataType.ARRAY:
+        assert schema.items is not None
+        return f"array[{display_type(schema.items, defs)}]"
+    elif st is None and (options := schema.anyOf) is not None:
+        schemas = [ref_to_schema(s, defs) for s in options]
+        return " | ".join(s.title or str(s.type.value) for s in schemas)  # type: ignore[union-attr]
+    else:
+        raise NotImplementedError(f"Type {st} not implemented. Schema: {schema!r}.")
+
+
+def italic(s: str) -> str:
+    return f"*{s}*"
+
+
+def bold(s: str) -> str:
+    return f"**{s}**"
+
+
+def mono(s: str) -> str:
+    return f"`{s}`"

docs/index.md

@@ -1,7 +1,60 @@
 # Synthesize
 
+Synthesize is a tool for managing long-lived development workflows that involve multiple tools executing concurrently,
+each of which might have bespoke conditions around when and how it needs to be run or re-run.
+
+In Synthesize, a **flow** is a graph (potentially disjoint) of **nodes**,
+each of which runs a **target** whenever one of that node's **triggers** activates.
+Synthesize has a wide variety of triggers:
+
+- Target `B` should run after target `A` runs.
+- Target `W` should run every time file `F` changes.
+- Target `R` should be restarted if it ever exits.
+- Target `O` should run once when the flow starts.
+
+These can all coexist as part of same flow, and can even be combined for a single target,
+allowing for complex nodes like
+["restart target `W` if it exits or if file `F` changes"](./triggers.md#example-restarting-on-completion-or-config-changes).
+
+## Features
+
+- Target and trigger definitions can be factored out and shared across multiple nodes and flows.
+- Targets are just shell commands, so you can use any tools you'd like. Synthesize works with your existing tools, it doesn't replace them.
+- Targets can be parameterized with arguments (each target is actually a [Jinja template](https://jinja.palletsprojects.com/)) and environment variables.
+  Arguments and environment variables can also be provided at the flow and target levels (most specific wins).
+- Nodes can have multiple triggers, allowing you to express complex triggering conditions.
+- All command output is combined in a single output stream, with each node's output prefixed with a timestamp and its name.
+- The current time and the status of each node is displayed at the bottom of your terminal.
+- You can generate [Mermaid](https://mermaid.js.org/) diagrams of your flows for debugging and documentation.
+
+## Examples
+
+As an example, here is Synthesize's own `synth.yaml` configuration file:
+
 ```yaml
 --8<-- "synth.yaml"
 ```
 
 @mermaid(synth.yaml)
+
+## Installation
+
+Synthesize is [available on PyPI](https://pypi.org/project/synthesize/).
+
+We recommend installing Synthesize via `pipx`:
+
+```bash
+pipx install synthesize
+```
+
+Then run
+```
+synth --help
+```
+to get started.
+
+## Inspirations
+
+- [`concurrently`](https://www.npmjs.com/package/concurrently)
+- [`make`](https://www.gnu.org/software/make/)
+- [`just`](https://github.com/casey/just)

docs/targets.md

@@ -0,0 +1,3 @@
+# Targets
+
+@schema(synthesize.config, Target)