docs: restructure the roadmap

.github/ISSUE_TEMPLATE/bug_report.md

@@ -1,28 +1,26 @@
 ---
 name: Bug report
 about: Create a report to help us improve
-title: ''
+title: ""
 labels: bug
-assignees: ''
-
+assignees: ""
 ---
 
-**Describe the bug**
+## What's wrong?
 
 A clear and concise description of what the bug is.
 
-**To Reproduce**
-
-Steps to reproduce the behavior:
-
-**Expected behavior**
+## Expected behavior
 
 A clear and concise description of what you expected to happen.
 
-**Screenshots**
+## Steps to reproduce
+
+Steps to reproduce the behavior:
 
-If applicable, add screenshots to help explain your problem.
+- Go to '...'
+- Run '....'
 
-**Additional context**
+## Additional context
 
 Add any other context about the problem here.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,19 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: ""
+labels: enhancement
+assignees: ""
+---
+
+## What
+
+A clear and concise description of what you want this feature to do.
+
+## Motivation
+
+A clear and concise description of the goals of the feature.
+
+## Open questions
+
+If any, list open questions here.

.github/ISSUE_TEMPLATE/support_request.md

@@ -0,0 +1,7 @@
+---
+name: Support request
+about: Ask a question about this project
+title: ""
+labels: question
+assignees: ""
+---

docs/docs/reference/language/spec.md

@@ -1,5 +1,11 @@
 # Specification
 
+:::warning
+
+This document is a work in progress. It's not complete and it's not final.
+
+:::
+
 The purpose of this document is to spec `TypeStream` language features.
 
 While `TypeStream` syntax is inspired by

docs/docs/roadmap.md

@@ -2,42 +2,80 @@
 
 A loose outlook of where `TypeStream` is headed.
 
-## Short term
+There's no particular order to this list of lists at the moment. This reflects
+the current state of ideas we may or may not implement in the near feature.
 
-- Language
-  - introduce functions and expression statements like:
-    - `answers=$(cat questions | grep answers); grep $answers 42 > result_42; grep $answers 24 > result_24`
-- Commands (these imply supporting more types)
-  - find
-  - tr
-- Serialization
-  - extend support for `protobuf`
-  - Add JSON schema encoding support
+To get a better sense of the upcoming features, check out the
+[issues](https://github.com/typestreamio/typestream/issues).
 
-## Medium term
+## Language
 
-- Introduce new sources:
-  - pulsar
-  - postgresql
-- Introduce naming strategies for: graph nodes, `TypeStream` apps.
+Refer to the [specs](reference/language/spec.md) for the current state of the language.
 
-## Long term
+- introduce functions and expression statements like:
+  - `answers=$(cat questions | grep answers); grep $answers 42 > result_42; grep $answers 24 > result_24`
+- support for `*` expansion
+- support list values (it's necessary for the `*` expansion and commands like find and tr)
+- support for DataStream Values (commands like find and ls imply this)
 
-- Introduce a [job planner](#job-planner).
-- Virtual filesystem
-  - Introduce setting for displaying topics with conventions (vs code uses
-    patterns for folders and maybe it's more flexible). It has to be cluster
-    specific.
-  - `/media` filesystem so we can expose data like `/media/websocket/server1`
+## New commands
 
-### Job planner
+Since it's early days, some commands will require work at the language level but
+we still have a separate section for them so it's clearer what commands we know
+we want to support.
 
-It would be interesting to have a streaming job planner. Here's an example:
+- find
+- tr
+- [tee](https://github.com/typestreamio/typestream/issues/78)
+- mount (see [Filesystem](#filesystem))
 
-```sh
-grep topic answer > result # first job
-grep topic answer > another.topic # second job
-```
+## Filesystem
 
-obviously it feels like we can reuse `grep topic answer`. Can we? If so, the
-planner would be responsible to answer this question.
+The virtual filesystem is a core component of `TypeStream`. It's how the
+metaphor "everything is a stream" is implemented.
+
+- `/media` filesystem (aka "mounting")
+  - websocket server
+  - http server
+  - jdbc server
+- There's no way to get the output of a long running program. We want to make
+  this work [How to view the output of a running process in another bash
+  session?](https://unix.stackexchange.com/questions/58550/how-to-view-the-output-of-a-running-process-in-another-bash-session)
+  at filesystem level
+
+## CI
+
+We could always have more tests... right? 😃
+
+- integration tests (defined as cli + server interactions. The whole stack only
+  basically)
+  - kubernetes features
+    - config map
+    - need to test the server and worker at integration level
+  - This test suite must be fully integrated with the release process (as we
+    want to run it on release branches automatically. Should also be possible to
+    run manually or trigger in pull requests)
+
+## Server
+
+- Failure handling strategy
+  - we have to restart watchers if they fail (retry until a certain threshold
+    and then crash the server?)
+  - if the catalog fails, we can't type check but the errors are unclear (as we
+    just say we can't find the type)
+  - Add an exception handler for kafka streams applications. You don't want them
+    to blow up but to log stuff
+
+## Encoding
+
+- schema enum isn't really efficient (as it carries the type in each instance, we need a container type for this)
+- [Avro](https://avro.apache.org)
+  - support duration
+  - some types metadata isn't carried over (see logical types with precision and scale for example)
+- [Protocol Buffers](https://protobuf.dev/)
+  - Introduce "SmokeType" first so we can support:
+    - google imports
+    - nested types
+    - multiple messages in the same definition
+- [json schema](https://json-schema.org)
+- [cloud events](https://cloudevents.io)