Skip to content

docs: better explain the architecture and more clearly separate docker setup from lndmon setup (mermaid version) #124

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
djkazic merged 1 commit into from
Dec 18, 2025
Merged

docs: better explain the architecture and more clearly separate docker setup from lndmon setup (mermaid version) #124

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 94 additions & 0 deletions Docker_Usage.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
# Docker Compose Setup Instructions for Collecting Data From `lnd` _AND_ `lndmon`

# Table of Contents
1. [Setup](#setup)
1. [Requirements](#requirements)
2. [Lnd](#lnd)
3. [Lndmon](#basic-lndmon-and-docker-compose-setup)
2. [Usage](#usage)
1. [Option 1: Nginx Proxy Usage](#option-1-nginx-proxy-usage-requires-domain-name)
2. [Option 2: Local Usage](#option-2-local-usage)
3. [Customizing Grafana Metrics](#customizing-grafana-metrics)

## Setup

### Requirements
* docker >= 18.09.6
* docker-compose >= 1.24

### Lnd

- You need to have an `lnd` node built with the `monitoring` tag up and running
with ports exposed and reachable.
- Run lnd with
* `prometheus.enable=true`
* `prometheus.listen=<IP>:8989`
* `rpclisten=<IP>`
* `tlsextraip=<IP>`
- Make sure you have the node's reachable IP defined in the cert.
- You may need to delete the existing `tls.cert` and `tls.key` in your lnd
data directory and restart lnd to regenerate the cert.
- Start `lnd` *before* `lndmon`.


Note: See README.md for some additional consideration about the options that
are set above.


### Basic `lndmon` and Docker Compose Setup

- Clone this repo
```
git clone https://github.com/lightninglabs/lndmon
cd lndmon/
```
- In the `.env` file in this repo
* Fill in the `TLS_CERT_PATH` and `MACAROON_PATH` variables relative to the
host filesystem to allow `lndmon` to connect to your `lnd` node.
- By default, `.lnd` lives in your home directory.
* Fill in the `LND_HOST` variable to match your lnd node's IP and port.
* Ensure the other lnd variables are also up-to-date.
* If you wish to run `lndmon` connecting to an `lnd` node on testnet or simnet:
- modify the `LND_NETWORK` variable to match your desired network.
- make sure the `MACAROON_PATH` matches the desired network as well.
- Edit `prometheus.yml` `lnd` `targets` section to match your node's IP.



## Usage
### Option 1: Nginx Proxy Usage (requires domain name)
If you want to enable the built-in nginx proxy feature in order to access your Prometheus and Grafana dashboards remotely, these are the steps:
1. In the `lndmon` repository, edit the `.env` file and fill in the email, FQDN, and (optionally) timezone fields.
2. Ensure ports 80 and 443 on your machine are exposed to the internet.
3. (Optional) Basic auth setup for your Prometheus dashboard:
- Install `apache2-utils` package.
- Run `htpasswd -c nginx/etc/.htpasswd <YOUR_USERNAME>` and follow the prompts to enter and confirm your desired password.
- In `lndmon/nginx/etc/service.conf`, uncomment the lines indicated in the file to enable basic auth.
4. If you want to use your own TLS certs:
- Uncomment the lines beginning with `SSL_` in `.env` and fill in the paths to your cert files.
- Uncomment the lines beginning with `- SSL_` in `docker-compose.nginx.yml`.
5. Start everything up
- `docker-compose -f docker-compose.yml -f docker-compose.nginx.yml up`
- Run this command within the `lndmon` repository directory
- Unless you opted to use your own certs above, this will result in the automatic generation of TLS certificates through Let's Encrypt if they haven't been generated already, or their renewal if the current certs have expired. The certs will automatically renew when they expire.
6. Grafana is located at `https://<YOUR_DOMAIN>/grafana/`
7. Prometheus's expression browser is located at `https://<YOUR_DOMAIN>/prometheus/graph`.

### Option 2: Local Usage
1. Start everything up
- `docker-compose up`
* Run this command within the `lndmon` repository directory
* If you get the error "transport: Error while dialing dial tcp 172.17.0.1:10009: i/o timeout", your docker interface may not have the default IP. Make sure your docker interface's IP matches the IP for `LND_HOST` in `.env` and the lnd target's IP in `prometheus.yml`.
2. Access Grafana dashboard:
- Get Grafana's IP:
- `docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' lndmon_grafana_1`
- Grafana's dashboard is located at `http://<GRAFANA_IP>:3000/`.
- The default password for the admin user is also admin (you can change it after the first login).
3. Access Prometheus expression browser:
- Get Prometheus's IP:
-`docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' lndmon_prometheus_1`
- Prometheus's expression browser is located at `http://<PROMETHEUS_IP>:9090/graph`.


### Customizing Grafana Metrics
`lndmon`'s Grafana instance comes with a set of basic dashboards. Add additional dashboards by clicking the `+' sign on the left.
88 changes: 0 additions & 88 deletions INSTALL.md
View file
Open in desktop

This file was deleted.

181 changes: 157 additions & 24 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,38 +1,117 @@
# lndmon

#### A drop-in monitoring solution for your lnd node using Prometheus and Grafana.
#### A drop-in monitoring solution for your lnd node that integrates with Prometheus and Grafana.

<img src="dashboard.png">

## What is this?

`lndmon` is a drop-in, dockerized monitoring/metric collection solution for your
individual lnd nodes connected to bitcoin. With this system, you'll be able to
closely monitor the health and status of your `lnd` node.
`lndmon` is a drop-in, monitoring/metric collection solution for your
individual lnd nodes connected to bitcoin which allows you to closely monitor
the health and status of your `lnd` node.

There are three primary components of the `lndmon` system:
There are four primary components of this monitoring system:

1. [lnd](https://github.com/lightningnetwork/lnd) built with the `monitoring`
tag, which enables lnd to export metrics about its gRPC performance and
usage. These metrics provide insights such as how many bytes lnd is
transmitting over gRPC, whether any calls are taking a long time to complete,
and other related statistics.

2. `lndmon`: while lnd provides some information, `lndmon` by far does the heavy
lifting with regards to metrics. With `lndmon`'s data, you can track routing
fees over time, track how the channel graph evolves, and have a highly
configurable "crystal ball" to forecast and de-escalate potential issues as
the network changes over time. There is also a strong set of metrics for
users who want to keep track of their own node and channels, or just explore
and create their own lightning data visualizations.

3. Last but not least, `lndmon` uses [Grafana](https://grafana.com/) as its
tag, which enables lnd to export metrics about its gRPC performance and
usage.
- These metrics provide insights such as how many bytes lnd is transmitting
over gRPC, whether any calls are taking a long time to complete, and other
related statistics.
- `lnd` build notes:
* If you are using a release build of `lnd` or `litd`, this was enabled
during build and no action is required.
* If you are building from source, make sure to include the `monitoring`
tag (i.e. `make && make install tags=monitoring`)
- Default `lnd` config options:
* `prometheus.enable=false`
- Need to change to `true` to enable.
* `prometheus.listen=127.0.0.1:8989`
- You may want to change to `0.0.0.0:8989` to listen on all interfaces.
* `prometheus.perfhistograms=false`
- Enabling this uses more RAM and disk space.
- Note: This data can be obtained from `lnd` without `lndmon` being
installed.

2. `lndmon`: while lnd provides some information about gRPC performance
statistics directly via its own prometheus exporter, `lndmon` by far does
the heavy lifting with regards to metrics.
- With `lndmon`'s data, you can track routing fees over time, track how the
channel graph evolves, and have a highly configurable "crystal ball" to
forecast and de-escalate potential issues as the network changes over time.
- There is also a strong set of metrics for users who want to keep track of
their own node and channels, or just explore and create their own
lightning data visualizations.
- See [Options](#options) below for `lndmon` config options.
- Note: This data can be obtained without lnd being built with the
`monitoring` tag and without the `prometheus.enable` config option enabled,
but it does require `lnd` to be listening for gRPC connections.

3. Prometheus is a time series database that scrapes data from `lnd` and
`lndmon` and stores it for later analysis.

4. `lndmon` uses [Grafana](https://grafana.com/) as its
primary dashboard to display all its collected metrics. Grafana is highly
configurable and can create beautiful and detailed graphs organized by
category (i.e., chain-related graphs, fee-related graphs, etc). Users have
the option of making their Grafana dashboards remotely accessible over TLS
with passwords to ensure their data is kept private.


# Architectural Layout
```mermaid
---
config:
flowchart:
subGraphTitleMargin:
bottom: 30
---
flowchart
subgraph s1["**LND**"]
n15@{ shape: "stadium", label: "gRPC Server \n(TCP/10009)" }
n20@{ shape: "stadium", label: "Prometheus Exporter \n (TCP/8989) " }
end
s1

n15
subgraph s2["**LNDmon**
(aggregate, cache)"]

n17@{ shape: "stadium", label: "Prometheus Exporter \n (TCP/9092)" }
n16@{ shape: "stadium", label: "gRPC Client \n (TCP/9090)" }
end
n16@{ shape: "stadium", label: "gRPC Client<br>(TCP/9090)" }
n21@{ shape: "fr-rect", label: "gRPC Performance Stats" }
n20@{ shape: "stadium", label: "Prometheus Exporter<br>(TCP/8989)" } --- n21
subgraph s3["**Prometheus**<br/>(time series DB)"]
n23@{ shape: "stadium", label: "HTTP Server" }
n22@{ shape: "stadium", label: "Prometheus Scraper" }
end
n21 --- n22
n7@{ shape: "fr-rect", label: "gRPC Call Data" }
n17@{ shape: "stadium", label: "Prometheus Exporter<br>(TCP/9092)" } --- n7
n7 --- n22
subgraph s4["**Grafana**<br/>(dashboard generation)"]
n25@{ shape: "stadium", label: "HTTP Server" }
n24@{ shape: "stadium", label: "HTTP Client" }
end
subgraph s5["**Firefox**<br/>(dashboard display)"]
n26@{ shape: "stadium", label: "HTTP Client" }
end
n23@{ shape: "stadium", label: "HTTP Server<br>(TCP/9090)" } --- n24
n25@{ shape: "stadium", label: "HTTP Server<br>(TCP/3000)" } --- n26
n15 --- n16@{ shape: "stadium", label: "gRPC Client" }
s1
n21
n22 --- n23
n24 --- n25
n16 --- n17
```





## Why would I want to use this?

Monitoring can provide crucial insights into the health of large-scale
Expand All @@ -46,9 +125,63 @@ ecosystem (i.e., "wow, the day [Lightning
App](https://github.com/lightninglabs/lightning-app) was released coincides with
the addition of 3000 channels to the network!").

## How do I install this?

Head over to [`INSTALL.md`](https://github.com/lightninglabs/lndmon/blob/master/INSTALL.md). It
also includes instructions to set up, access, and password-protect the dashboard
that comes with Prometheus, called the Prometheus expression browser, for those
interested in using it.
## Build

```
git clone https://github.com/lightninglabs/lndmon
cd lndmon/cmd/lndmon
go build
```

## Options

```
$ lndmon -h
Usage:
lndmon [OPTIONS]

Application Options:
--primarynode= Public key of the primary node in a primary-gateway setup
--disablegraph Do not collect graph metrics
--disablehtlc Do not collect HTLCs metrics
--disablepayments Do not collect payments metrics

prometheus:
--prometheus.listenaddr= the interface we should listen on for prometheus (default:
localhost:9092)
--prometheus.logdir= Directory to log output (default: /home/user/.lndmon/logs)
--prometheus.maxlogfiles= Maximum log files to keep (0 for no rotation) (default: 3)
--prometheus.maxlogfilesize= Maximum log file size in MB (default: 10)

lnd:
--lnd.host= lnd instance rpc address (default: localhost:10009)
--lnd.network=[regtest|testnet|testnet4|mainnet|simnet|signet] network to run on (default: mainnet)
--lnd.macaroondir= Path to lnd macaroons (default: /home/user/.lnd)
--lnd.macaroonname= The name of our macaroon in macaroon dir to use. (default:
readonly.macaroon)
--lnd.rpctimeout= The timeout for rpc calls to lnd. Valid time units are {s,
m, h}. (default: 30s)
--lnd.tlspath= Path to lnd tls certificate

Help Options:
-h, --help Show this help message
```











## How do I use this?

Head over to [`Docker_Usage.md`](https://github.com/lightninglabs/lndmon/blob/master/Docker_Usage.md)
for information on how to setup a dockerized version. It also includes
instructions to set up, access, and password-protect the dashboard that comes
with Prometheus, called the "Prometheus Expression Browser", for those interested
in using it.
Loading