remove storage (the great simplification) (#134)

* remove storage (the great simplification)

* fix docker ci

* `--help`

* ci

* docs

* remove double log

* fix missing .source through types

* safeguard #emit

* typecheck metrics#submit

* types

* more types :)

* fix merge metrics logic

* refactor using `onActivity`

* refactor using `onMetrics`

* add passing test for merging metrics

* add passing test for metrics deduplication

* docs

Author: juliangruber

.github/workflows/ci.yml

@@ -89,10 +89,10 @@ jobs:
       - name: Print station activity
         run: |
           sleep 10 # Wait for Station modules to start
-          docker exec station bin/station.js activity
+          docker logs station
 
       - name: Check | Saturn started
-        run: docker exec station bin/station.js activity | grep "Saturn module started"
+        run: docker logs station | grep "Saturn module started"
 
       - name: Check | Zinnia started
-        run: docker exec station bin/station.js activity | grep "Zinnia started"
+        run: docker logs station | grep "Zinnia started"

README.md

@@ -122,127 +122,13 @@ Modules currently in experimental mode:
 
 - [Bacalhau](https://github.com/bacalhau-project/bacalhau)
 
-### `$ station metrics <module>`
-
-Get combined metrics from all Station Modules:
-
-```bash
-$ station metrics
-{
-	"totalJobsCompleted": 123,
-	"totalEarnings": "0"
-}
-```
-
-Get metrics from a specific Station Module:
-
-```bash
-$ station metrics saturn-l2-node
-{
-	"totalJobsCompleted": 123,
-	"totalEarnings": "0"
-}
-```
-
-Follow metrics:
-
-```bash
-$ station metrics --follow
-{
-	"totalJobsCompleted": 123,
-	"totalEarnings": "0"
-}
-...
-```
-
-### `$ station activity`
-
-Get Station activity logs.
-
-Get all activity:
-
-```bash
-$ station activity
-[3/14/2023, 10:23:14 AM] INFO  Saturn Node is online and connected to 1 peers
-[3/14/2023, 10:23:18 AM] INFO  Saturn Node is online and connected to 9 peers
-```
-
-```bash
-$ station activity --json
-[
-  {
-    "timestamp": "2023-04-05T11:26:41.000Z",
-    "type": "info",
-    "source": "Saturn",
-    "message": "Saturn Node is online and connected to 1 peers",
-    "id": "b6e922f2-54fb-4d5b-adcb-499391e4a09e"
-  },
-  {
-    "timestamp": "2023-04-05T11:26:46.000Z",
-    "type": "info",
-    "source": "Saturn",
-    "message": "Saturn Node is online and connected to 9 peers",
-    "id": "ffb551a0-247f-471d-b4db-0145cc3b1614"
-  }
-]
-```
-
-Follow activity:
-
-```bash
-$ station activity --follow
-[3/14/2023, 10:23:14 AM] INFO  Saturn Node is online and connected to 1 peers
-[3/14/2023, 10:23:18 AM] INFO  Saturn Node is online and connected to 9 peers
-...
-```
-
-```bash
-$ station activity --follow --json
-{"timestamp":"2023-04-05T11:26:41.000Z","type":"info","source":"Saturn","message":"Saturn Node is online and connected to 1 peers""id":"b6e922f2-54fb-4d5b-adcb-499391e4a09e"}
-{"timestamp":"2023-04-05T11:26:46.000Z","type":"info","source":"Saturn","message":"Saturn Node is online and connected to 9 peers""id":"ffb551a0-247f-471d-b4db-0145cc3b1614"}
-```
-
-### `$ station logs <module>`
-
-Get combined logs from Station Core and all Station Modules:
-
-```bash
-$ station logs
-[3/14/2023, 5:15:26 PM] [saturn-L2-node] INFO: Saturn Node is online and connected to 9 peers
-[3/14/2023, 5:15:26 PM] [saturn-L2-node] ERROR: Saturn Node is not able to connect to the network
-```
-
-Get logs from a specific Station Module:
-
-```bash
-$ station logs saturn-l2-node
-[3/14/2023, 5:15:26 PM] INFO: Saturn Node is online and connected to 9 peers
-[3/14/2023, 5:15:26 PM] ERROR: Saturn Node is not able to connect to the network
-```
-
-Follow logs:
-
-```bash
-$ station logs --follow
-[3/14/2023, 5:15:26 PM] [saturn-L2-node] INFO: Saturn Node is online and connected to 9 peers
-[3/14/2023, 5:15:26 PM] [saturn-L2-node] ERROR: Saturn Node is not able to connect to the network
-...
-```
-
 ### `$ station --help`
 
 Show help.
 
 ```bash
 $ station --help
-Usage: station <command> [options]
-
-Commands:
-  station                   Start Station                              [default]
-  station metrics [module]  Show metrics
-  station activity          Show activity log
-  station logs [module]     Show module logs
-  station events            Events stream
+Usage: station [options]
 
 Options:
   -j, --json          Output JSON                                      [boolean]
@@ -273,12 +159,6 @@ $ docker run \
 	ghcr.io/filecoin-station/core
 ```
 
-Inspect logs:
-
-```bash
-$ docker exec station ./bin/station.js logs
-```
-
 ## Manual Deployment (Ubuntu)
 
 On a fresh [Ubuntu](https://ubuntu.com/) machine:
@@ -314,57 +194,9 @@ EOF
 $ sudo systemctl daemon-reload
 $ sudo systemctl start station
 $ sudo systemctl status station
-```
-
-## API
-
-### `Core.create({ cacheRoot?: String, stateRoot?: String }?)`
-
-Returns `Promise<Core>`.
-
-### `Core#logs.get(module?: String)`
-
-Returns `Promise<String>`.
 
-### `Core#logs.follow(module?: String)`
-
-Returns `AsyncGenerator<String>`.
-
-### `Core#activity.get()`
-
-Returns `Promise<ActivityEvent[]>`.
-
-### `Core#activity.follow({ signal?: AbortSignal, nLines = 10?: Number }?)`
-
-Returns `AsyncGenerator<ActivityEvent>`.
-
-### `Core#metrics.getLatest(module?: String)`
-
-Returns `Promise<MetricsEvent>`.
-
-### `Core#metrics.follow({ module?: String, signal?: AbortSignal }?)`
-
-Returns `AsyncGenerator<MetricsEvent>`.
-
-### `ActivityEvent`
-
-```
-{
-  timestamp: Date
-  type: ("info"|"error")
-  source: String
-  message: String
-  id: String
-}
-```
-
-### `MetricsEvent`
-
-```
-{
-  totalJobsCompleted: Number
-  totalEarnings: String
-}
+# Read logs
+$ journalctl -u station.service
 ```
 
 ## Disclaimer

bin/station.js

@@ -2,14 +2,10 @@
 
 'use strict'
 
-const commands = require('../commands')
+const { station } = require('../commands/station')
 const Sentry = require('@sentry/node')
 const yargs = require('yargs/yargs')
 const { hideBin } = require('yargs/helpers')
-// We must not require('..') as that confuses TypeScript compiler.
-// The compiler will look at our package.json, find that the types are in `dist/index.d.ts`
-// and load that output file instead of the actual input `index.js`.
-const { Core } = require('../index')
 const pkg = require('../package.json')
 
 Sentry.init({
@@ -20,57 +16,24 @@ Sentry.init({
   ignoreErrors: [/EACCES/, /EPERM/, /ENOSPC/, /EPIPE/]
 })
 
-const main = async () => {
-  const core = await Core.create()
-
-  yargs(hideBin(process.argv))
-    .usage('Usage: $0 <command> [options]')
-    .command(
-      '$0',
-      'Start Station',
-      yargs => yargs
-        .option('json', {
-          alias: 'j',
-          type: 'boolean',
-          description: 'Output JSON'
-        })
-        .option('experimental', {
-          type: 'boolean',
-          description: 'Also run experimental modules'
-        }),
-      args => commands.station({ ...args, core })
-    )
-    .command(
-      'metrics [module]',
-      'Show metrics',
-      () => {},
-      ({ follow, module }) => commands.metrics({ core, follow, module })
-    )
-    .command(
-      'activity',
-      'Show activity log',
-      yargs => yargs.option('json', {
+yargs(hideBin(process.argv))
+  .usage('Usage: $0 [options]')
+  .command(
+    '$0',
+    'Start Station',
+    yargs => yargs
+      .option('json', {
         alias: 'j',
         type: 'boolean',
         description: 'Output JSON'
+      })
+      .option('experimental', {
+        type: 'boolean',
+        description: 'Also run experimental modules'
       }),
-      ({ follow, json }) => commands.activity({ core, follow, json })
-    )
-    .command(
-      'logs [module]',
-      'Show module logs',
-      () => {},
-      ({ module, follow }) => commands.logs({ core, module, follow })
-    )
-    .choices('module', core.modules)
-    .version(`${pkg.name}: ${pkg.version}`)
-    .alias('v', 'version')
-    .alias('h', 'help')
-    .alias('f', 'follow')
-    .parse()
-}
-
-main().catch(err => {
-  console.error(err)
-  process.exit(1)
-})
+    ({ json, experimental }) => station({ json, experimental })
+  )
+  .version(`${pkg.name}: ${pkg.version}`)
+  .alias('v', 'version')
+  .alias('h', 'help')
+  .parse()