Adding new Metrics mode to allow standalone or centralized storage (for DeployEx scenarios)

CHANGELOG.md

@@ -6,7 +6,7 @@
  * None
 
 ### Bug fixes
- * [[`PR-12`](https://github.com/thiagoesteves/observer_web/pull/12)] Fixing bug fix reporting that graphics were displaying information in the reverse order
+ * [[`PR-12`](https://github.com/thiagoesteves/observer_web/pull/12)] Fixing bug reported that graphics were displaying information in the reverse order
 
 ### Enhancements
  * [[`PR-10`](https://github.com/thiagoesteves/observer_web/pull/10)] Adding iframe configuration to allow Observer Web to run with embedded pages

README.md

@@ -38,12 +38,65 @@ mix assets.build
 
 Since Observer Web uses the OTP distribution, it is required to have a name when running.
 ```
-elixir --sname hello -S mix run --no-halt dev.exs
+elixir --sname observer -S mix run --no-halt dev.exs
 ```
 
 Now you can visit [`localhost:4000/observer`](http://localhost:4000/observer) from your browser.
 
 
+### Run multiple nodes with Metric Hub configuration
+
+1. Start the nodes
+
+Open a new terminal (Terminal 1) and run the application in `observer` mode:
+```
+export PORT=4000
+export OBSERVER_WEB_TELEMETRY_MODE=observer
+elixir --sname observer --cookie cookie -S mix run --no-halt dev.exs
+```
+
+Open a new terminal (Terminal 2) and run the application in `broadcast` mode:
+```
+export PORT=4001
+export OBSERVER_WEB_TELEMETRY_MODE=broadcast
+elixir --sname broadcast --cookie cookie -S mix run --no-halt dev.exs
+```
+
+2. Connect the nodes
+
+Open a new terminal (Terminal 3) and run:
+```
+iex --sname hub --cookie cookie -S mix phx.server
+```
+
+After the Phoenix server starts, you'll see the Elixir interactive shell prompt. Connect 
+the nodes by executing these commands:
+
+```elixir
+# Connect to the observer node
+{:ok, hostname} = :inet.gethostname()
+Node.connect(:"observer@#{hostname}")
+# Should return: true
+
+# Connect to the broadcast node
+{:ok, hostname} = :inet.gethostname()
+Node.connect(:"broadcast@#{hostname}")
+# Should return: true
+```
+
+you can close the Terminal 3 if you want, this terminal is only for supporting the node connection.
+
+To verify everything is working properly:
+
+ * Visit [`localhost:4000/observer/metrics`](http://localhost:4000/observer/metrics) in your browser to confirm 
+   the application is running in `observer` mode.
+ * Visit [`localhost:4001/observer/metrics`](http://localhost:4001/observer/metrics) to confirm the application 
+   is running in `broadcast` mode.
+
+You can now explore the `observer` mode, checking that the data is persisted even if the other app in
+broadcast mode restarts.
+
+
 [dye]: https://github.com/thiagoesteves/deployex
 [edb]: https://www.erlang.org/doc/apps/runtime_tools/dbg.html
 [liv]: https://github.com/phoenixframework/phoenix_live_view

config/config.exs

@@ -30,10 +30,6 @@ if config_env() == :dev do
       ),
       cd: Path.expand("../assets", __DIR__)
     ]
-
-  config :observer_web, ObserverWeb.Telemetry,
-    adapter: ObserverWeb.Telemetry.Storage,
-    data_retention_period: :timer.minutes(30)
 end
 
 # Configures Elixir's Logger

dev.exs

@@ -75,6 +75,22 @@ Application.put_env(:phoenix, :serve_endpoints, true)
 Application.put_env(:phoenix, :persistent, true)
 
 Task.async(fn ->
+  # Stop the default Telemetry server to start a new one with new defaults
+  mode = "OBSERVER_WEB_TELEMETRY_MODE" |> System.get_env("local") |> String.to_atom()
+
+  retention_period =
+    "OBSERVER_WEB_TELEMETRY_RETENTION_PERIOD" |> System.get_env("1800000") |> String.to_integer()
+
+  telemetry_module = ObserverWeb.Telemetry.Storage
+  :ok = Supervisor.terminate_child(ObserverWeb.Application, telemetry_module)
+  :ok = Supervisor.delete_child(ObserverWeb.Application, telemetry_module)
+
+  {:ok, _} =
+    Supervisor.start_child(
+      ObserverWeb.Application,
+      {telemetry_module, [mode: mode, data_retention_period: retention_period]}
+    )
+
   {:ok, _} = Supervisor.start_child(ObserverWeb.Application, WebDev.Endpoint)
 
   Process.sleep(:infinity)

guides/installation.md

@@ -54,23 +54,6 @@ After you've verified that the dashboard is loading you'll probably want to rest
 dashboard via authentication, either with a [custom resolver's][ac] access controls or [Basic
 Auth][ba].
 
-### Retention period for metrics
-
-The Observer Web can monitor Beam VM metrics by default, using ETS tables to store the data.
-However, this means that the data is not persisted across restarts. The retention period
-for this data can be configured.
-
-By default, without a retention time set, the metrics will only show data received during the
-current session. If you'd like to persist this data for a longer period, you can configure
-a retention time.
-
-To configure the retention period, use the following optional setting:
-
-```elixir
-config :observer_web, ObserverWeb.Telemetry,
-  data_retention_period: :timer.minutes(5)
-```
-
 ### Embedding Observer Web in your app page
 
 In some cases, you may prefer to run the Observer in the same page as your app rather than in 
@@ -139,6 +122,81 @@ path `/observer"`. However, using the iframe approach allows you to display
 your application's information alongside the Observer in your main page, 
 providing a more integrated monitoring experience.
 
+### Metrics
+
+#### Retention period for metrics
+
+Observer Web can monitor Beam VM metrics (along with many others) and uses ETS 
+tables to store the data and there is a possibility of configuration for the retention
+period while the application is running.
+
+By default, without a retention time set, the metrics will only show data received during the
+current session. If you'd like to persist this data for a longer period, you can configure
+a retention time.
+
+To configure the retention period, use the following optional setting:
+
+```elixir
+config :observer_web, ObserverWeb.Telemetry,
+  data_retention_period: :timer.minutes(30)
+```
+
+> #### Persistence Across Restarts {: .attention}
+>
+> Please note that data is not persisted across application restarts. For persistent
+> storage, refer to the Configuration section to set up a Central Hub application,
+> which can aggregate and retain metrics.
+
+#### Configuration
+
+Observer Web can operate in two distinct metrics configurations: `Standalone` and `Metric Hub`.
+These configurations determine how metrics are collected, stored, and managed.
+
+#### Standalone Configuration (default)
+
+In this mode, all applications with Observer Web installed operate independently. Each
+application receives and stores its own metrics within its ETS tables. The image below
+illustrates this configuration:
+
+![Standalone Mode](./static/standalone.png)
+
+__**NOTE: No additional configuration is required for this mode**__
+
+#### Metric Hub Configuration
+
+In this mode, one application is designated as the central hub to store all metrics,
+while the remaining applications broadcast their data to this designated hub. This
+configuration is ideal for scenarios where you have a dedicated application for monitoring
+or deployment, such as [DeployEx][dye]. Additionally, this setup ensures that metrics
+are retained even if any of the monitored applications restart.
+
+![Metric Hub Mode](./static/metric_hub.png)
+
+To configure applications to broadcast their metrics, use the following setting:
+
+```elixir
+config :observer_web, ObserverWeb.Telemetry,
+  mode: :broadcast
+```
+
+> #### Disable Endpoint for Broadcast applications {: .attention}
+>
+> In this mode, since there is a centralized application dedicated to capturing metrics,
+> we recommend disabling the `/observer` endpoint on all applications configured in 
+> **broadcast** mode. Only the **central observer (hub)** should expose the `/observer` endpoint
+> to avoid redundancy and ensure efficient metric collection.
+
+To designate an application as the **central observer (hub)**, use the following setting:
+
+```elixir
+config :observer_web, ObserverWeb.Telemetry,
+  mode: :observer,
+  data_retention_period: :timer.minutes(30)
+```
+
+The application in `observer mode` will also retain its own metrics in addition to 
+aggregating metrics from other applications.
+
 ### Usage with Web and Clustering
 
 The Observer Web provides observer ability for the local application as well as any other that is
@@ -158,3 +216,4 @@ via OTP distribution!
 [ac]: Observer.Web.Resolver.html#c:resolve_access/1
 [ba]: https://hexdocs.pm/basic_auth/readme.html
 [oi]: installation.html
+[dye]: https://github.com/thiagoesteves/deployex

guides/overview.md

@@ -8,6 +8,8 @@ and Beam VM statistics.
 Powered by [Phoenix LiveView][liv], it is distributed, lightweight, and fully real-time. This 
 library is part of the [DeployEx][dye] project.
 
+![Observer Dashboard](./static/dashboard.png)
+
 [dye]: https://github.com/thiagoesteves/deployex
 [edb]: https://www.erlang.org/doc/apps/runtime_tools/dbg.html
 [liv]: https://github.com/phoenixframework/phoenix_live_view