-
-
{{ .Title}}
-
- {{ .Params.bref }}
-
-
-
-
- {{ range .Data.Pages.ByWeight }}
-
-
-
{{ .Params.description }}
-
- {{ end }}
-
-
-
-
You didn't find Pub/Sub that you need? Check out the list of
- unofficial Pub/Subs.
-
-
-
If you want to find out how to implement your own Pub/Sub adapter, check out
- Implementing custom Pub/Sub.
-
-
+ {{ range $index, $tab := .Scratch.Get $group -}}
+
+ {{ .Content | $.Page.RenderString -}}
+
+ {{ end -}}
+
+
-#### Importing the dashboard
+### Importing the dashboard
To import the Grafana dashboard, select Dashboard/Manage from the left menu, and then click on `+Import`.
@@ -87,7 +72,7 @@ Enter the dashboard URL https://grafana.com/dashboards/9777 (or just the ID, 977
Then select your the Prometheus data source that scrapes the `/metrics` endpoint. Click on `Import`, and you're done!
-### Exported metrics
+## Exported metrics
Listed below are all the metrics that are registered on the Prometheus Registry by `PrometheusMetricsBuilder`.
@@ -139,15 +124,13 @@ Additionally, every metric has the `node` label, provided by Prometheus, with va
**NOTE**: As described [above](#wrapping-publishers-subscribers-and-handlers), using non-empty `namespace` or `subsystem` will result in prefixed metric names. You might need to adjust for it, for example in the definitions of panels in the Grafana dashboard.
-### Customization
+## Customization
If you feel like some metric is missing, you can easily expand this basic implementation. The best way to do so is to use the prometheus registry that is used with the [ServeHTTP method](#exposing-the-metrics-endpoint) and register a metric according to [the documentation](https://godoc.org/github.com/prometheus/client_golang/prometheus) of the Prometheus client.
An elegant way to update these metrics would be through the use of decorators:
-{{% render-md %}}
{{% load-snippet-partial file="src-link/message/decorator.go" first_line_contains="// MessageTransformSubscriberDecorator" last_line_contains="type messageTransformSubscriberDecorator" %}}
-{{% /render-md %}}
and/or [router middlewares](/docs/messages-router/#middleware).
diff --git a/docs/content/development/contributing.md b/docs/content/development/contributing.md
new file mode 100644
index 000000000..944f992e3
--- /dev/null
+++ b/docs/content/development/contributing.md
@@ -0,0 +1,53 @@
++++
+title = "Contributing Guide"
+description = "Contribute to Watermill"
+weight = 100
+bref = "Contribute to Watermill"
++++
+
+## How can I help?
+
+We are always happy to help you in contributing to Watermill. If you have any ideas, please let us know on our [Discord server](https://watermill.io/support/).
+
+There are multiple ways in which you can help us.
+
+### Existing issues
+
+You can pick one of the existing issues. Most of the issues should have an estimation (S - small, M - medium, L - large).
+
+- [Good first issues list](https://github.com/ThreeDotsLabs/watermill/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) - simple issues to begin with
+- [Help wanted issues list](https://github.com/ThreeDotsLabs/watermill/issues?q=is%3Aissue+is%3Aopen+label%3A%22help+wanted%22) - tasks that are already more or less clear, and you can start to implement them pretty quickly
+
+### New Pub/Sub implementations
+
+If you have an idea to create a Pub/Sub based on some technology and it is not listed yet in our issues (because we don't know it, or it is just some crazy idea, like physical mail based Pub/Sub), feel free to add your own implementation.
+
+You can do it in your private repository and later, if you want, we can move it to `ThreeDotsLabs/watermill-[name]`.
+You will keep the maintainer permissions to the repository, and we'll invite you to a maintainers-only discord channel.
+
+When adding a new Pub/Sub implementation, you should start with [Implementing a new Pub/Sub]({{< ref "pub-sub-implementing" >}}).
+
+### New ideas
+
+If you have any idea that is not covered in the issues list, please post a new issue describing it.
+It's recommended to discuss your idea on [Discord](https://discord.gg/QV6VFg4YQE)/GitHub before creating production-ready implementation - in some situations, it may save a lot of your time before implementing something that can be simplified or done more easily. :)
+
+In general, it's helpful to discuss a Proof of Concept to align with the idea.
+
+## Local development
+
+Makefile and docker-compose (for Pub/Subs) are your friends. You can run all tests locally (they are running in CI in the same way).
+
+Useful commands:
+- `make up` - docker-compose up
+- `make test` - tests
+- `make test_short` - run short tests (useful to perform a very fast check after changes)
+- `make fmt` - do goimports
+
+## Code standards
+
+- you should run `make fmt`
+- [CodeReviewComments](https://github.com/golang/go/wiki/CodeReviewComments)
+- [Effective Go](https://golang.org/doc/effective_go.html)
+- SOLID
+- code should be open for configuration and not coupled to any serialization method (for example: [AMQP marshaler](https://github.com/ThreeDotsLabs/watermill-amqp/blob/master/pkg/amqp/marshaler.go), [AMQP Config](https://github.com/ThreeDotsLabs/watermill-amqp/blob/master/pkg/amqp/config.go)
diff --git a/docs/content/docs/pub-sub-implementing.md b/docs/content/development/pub-sub-implementing.md
similarity index 92%
rename from docs/content/docs/pub-sub-implementing.md
rename to docs/content/development/pub-sub-implementing.md
index 6531014ae..71e4827b5 100644
--- a/docs/content/docs/pub-sub-implementing.md
+++ b/docs/content/development/pub-sub-implementing.md
@@ -1,28 +1,25 @@
+++
-title = "Implementing custom Pub/Sub"
+title = "Implementing a new Pub/Sub"
description = "Bring Your Own Pub/Sub"
date = 2018-12-05T12:48:34+01:00
-weight = -300
+weight = 200
draft = false
bref = "Bring Your Own Pub/Sub"
-toc = true
+++
-### The Pub/Sub interface
+## The Pub/Sub interface
To add support for a custom Pub/Sub, you have to implement both `message.Publisher` and `message.Subscriber` interfaces.
-{{% render-md %}}
{{% load-snippet-partial file="src-link/message/pubsub.go" first_line_contains="type Publisher interface" last_line_contains="type SubscribeInitializer" padding_after="0" %}}
-{{% /render-md %}}
-### Testing
+## Testing
Watermill provides [a set of test scenarios](https://github.com/ThreeDotsLabs/watermill/blob/master/pubsub/tests/test_pubsub.go)
that any Pub/Sub implementation can use. Each test suite needs to declare what features it supports and how to construct a new Pub/Sub.
These scenarios check both basic usage and more uncommon use cases. Stress tests are also included.
-### TODO list
+## TODO list
Here are a few things you shouldn't forget about:
diff --git a/docs/content/development/releases.md b/docs/content/development/releases.md
new file mode 100644
index 000000000..9d11ad429
--- /dev/null
+++ b/docs/content/development/releases.md
@@ -0,0 +1,10 @@
++++
+title = "Releases"
+description = "Watermill Releases"
+weight = 200
+bref = "Watermill Releases"
++++
+
+You can read about the historical Watermill releases in [the posts on the Three Dots Labs blog](https://threedots.tech/series/watermill-release-post/).
+
+
diff --git a/docs/content/docs/_index.md b/docs/content/docs/_index.md
new file mode 100644
index 000000000..afc39df94
--- /dev/null
+++ b/docs/content/docs/_index.md
@@ -0,0 +1,15 @@
+---
+title: "Docs"
+description: ""
+summary: ""
+date: 2023-09-07T16:12:03+02:00
+lastmod: 2023-09-07T16:12:03+02:00
+draft: false
+weight: 999
+toc: true
+seo:
+ title: "" # custom title (optional)
+ description: "" # custom description (recommended)
+ canonical: "" # custom canonical URL (optional)
+ noindex: false # false (default) or true
+---
diff --git a/docs/content/docs/articles.md b/docs/content/docs/articles.md
new file mode 100644
index 000000000..5fbde739d
--- /dev/null
+++ b/docs/content/docs/articles.md
@@ -0,0 +1,15 @@
++++
+title = "Articles"
+description = "In-depth articles mentioning Watermill"
+weight = 100
+draft = false
+bref = "In-depth articles mentioning Watermill"
++++
+
+You can find more in-depth tips on Watermill in these articles:
+
+* [Distributed Transactions in Go: Read Before You Try](https://threedots.tech/post/distributed-transactions-in-go/)
+* [Live website updates with Go, SSE, and htmx](https://threedots.tech/post/live-website-updates-go-sse-htmx/)
+* [Using MySQL as a Pub/Sub](https://threedots.tech/post/when-sql-database-makes-great-pub-sub/)
+* [Creating local Go dev environment with Docker and live code reloading](https://threedots.tech/post/go-docker-dev-environment-with-go-modules-and-live-code-reloading/)
+
diff --git a/docs/content/docs/awesome.md b/docs/content/docs/awesome.md
index 885cd614a..7f7e39890 100644
--- a/docs/content/docs/awesome.md
+++ b/docs/content/docs/awesome.md
@@ -1,11 +1,9 @@
+++
title = "Awesome Watermill"
description = "Selected unofficial libraries"
-weight = 0
+weight = 200
draft = false
-toc = true
bref = "Selected unofficial libraries"
-type = "docs"
+++
Below is a list of libraries that are not maintained by Three Dots Labs, but you may find them useful.
@@ -14,13 +12,13 @@ Below is a list of libraries that are not maintained by Three Dots Labs, but you
If you know another library or are an author of one, please [add it to the list](https://github.com/ThreeDotsLabs/watermill/edit/master/docs/content/docs/awesome.md).
-### Examples
+## Examples
* https://github.com/minghsu0107/golang-taipei-watermill-example
* https://github.com/minghsu0107/Kafka-PubSub
* https://github.com/pperaltaisern/go-example-financing
-### Pub/Subs
+## Pub/Subs
* AMQP 1.0 https://github.com/kahowell/watermill-amqp10
* Apache Pulsar https://github.com/AlexCuse/watermill-pulsar
@@ -35,9 +33,9 @@ If you know another library or are an author of one, please [add it to the list]
* SQLite https://github.com/davidroman0O/watermill-comfymill
If you want to find out how to implement your own Pub/Sub adapter,
-check out [Implementing custom Pub/Sub](/docs/pub-sub-implementing).
+check out [Implementing custom Pub/Sub](/development/pub-sub-implementing).
-### Logging
+## Logging
* logrus
* https://github.com/ma-hartma/watermill-logrus-adapter
@@ -51,7 +49,7 @@ check out [Implementing custom Pub/Sub](/docs/pub-sub-implementing).
* https://github.com/bogatyr285/watermillzlog
* https://github.com/vsvp21/zerolog-watermill-adapter
-### Observability
+## Observability
* OpenCensus
* https://github.com/czeslavo/watermill-opencensus
@@ -62,7 +60,7 @@ check out [Implementing custom Pub/Sub](/docs/pub-sub-implementing).
* AMQP https://github.com/hpcslag/otel-watermill-amqp
* GoChannel https://github.com/hpcslag/watermill-otel-tracable-gochannel
-### Other
+## Other
* https://github.com/asyncapi/go-watermill-template
* https://github.com/goph/watermillx
diff --git a/docs/content/docs/cqrs.md b/docs/content/docs/cqrs.md
index 81e91bd4b..4729dc7e6 100644
--- a/docs/content/docs/cqrs.md
+++ b/docs/content/docs/cqrs.md
@@ -4,8 +4,7 @@ description = "Build CQRS and Event-Driven applications"
date = 2019-02-12T12:47:30+01:00
weight = -400
draft = false
-bref = "Golang CQRS implementation in Watermill"
-toc = true
+bref = "Go CQRS implementation in Watermill"
+++
## CQRS
@@ -16,7 +15,7 @@ toc = true
>
> Source: [www.cqrs.nu FAQ](http://www.cqrs.nu/Faq/command-query-responsibility-segregation)
-
+
The `cqrs` component provides some useful abstractions built on top of Pub/Sub and Router that help to implement the CQRS pattern.
@@ -30,41 +29,27 @@ The event represents something that already took place. Events are immutable.
#### Event Bus
-{{% render-md %}}
{{% load-snippet-partial file="src-link/components/cqrs/event_bus.go" first_line_contains="// EventBus" last_line_contains="type EventBus" padding_after="0" %}}
-{{% /render-md %}}
-{{% render-md %}}
{{% load-snippet-partial file="src-link/components/cqrs/event_bus.go" first_line_contains="type EventBusConfig" last_line_contains="func (c *EventBusConfig) setDefaults()" padding_after="4" %}}
-{{% /render-md %}}
#### Event Processor
-{{% render-md %}}
{{% load-snippet-partial file="src-link/components/cqrs/event_processor.go" first_line_contains="// EventProcessor" last_line_contains="type EventProcessor" padding_after="0" %}}
-{{% /render-md %}}
-{{% render-md %}}
{{% load-snippet-partial file="src-link/components/cqrs/event_processor.go" first_line_contains="type EventProcessorConfig" last_line_contains="func (c *EventProcessorConfig) setDefaults()" padding_after="4" %}}
-{{% /render-md %}}
#### Event Group Processor
-{{% render-md %}}
{{% load-snippet-partial file="src-link/components/cqrs/event_processor_group.go" first_line_contains="// EventGroupProcessor" last_line_contains="type EventGroupProcessor" padding_after="0" %}}
-{{% /render-md %}}
-{{% render-md %}}
{{% load-snippet-partial file="src-link/components/cqrs/event_processor_group.go" first_line_contains="type EventGroupProcessorConfig" last_line_contains="func (c *EventGroupProcessorConfig) setDefaults()" padding_after="4" %}}
-{{% /render-md %}}
Learn more in [Event Group Processor](#event-handler-groups).
#### Event Handler
-{{% render-md %}}
{{% load-snippet-partial file="src-link/components/cqrs/event_handler.go" first_line_contains="// EventHandler" last_line_contains="type EventHandler" padding_after="0" %}}
-{{% /render-md %}}
#### Command
@@ -72,35 +57,23 @@ The command is a simple data structure, representing the request for executing s
#### Command Bus
-{{% render-md %}}
{{% load-snippet-partial file="src-link/components/cqrs/command_bus.go" first_line_contains="// CommandBus" last_line_contains="type CommandBus" padding_after="0" %}}
-{{% /render-md %}}
-{{% render-md %}}
{{% load-snippet-partial file="src-link/components/cqrs/command_bus.go" first_line_contains="type CommandBusConfig" last_line_contains="func (c *CommandBusConfig) setDefaults()" padding_after="4" %}}
-{{% /render-md %}}
#### Command Processor
-{{% render-md %}}
{{% load-snippet-partial file="src-link/components/cqrs/command_processor.go" first_line_contains="// CommandProcessor" last_line_contains="type CommandProcessor" padding_after="0" %}}
-{{% /render-md %}}
-{{% render-md %}}
{{% load-snippet-partial file="src-link/components/cqrs/command_processor.go" first_line_contains="type CommandProcessorConfig" last_line_contains="func (c *CommandProcessorConfig) setDefaults()" padding_after="4" %}}
-{{% /render-md %}}
#### Command Handler
-{{% render-md %}}
{{% load-snippet-partial file="src-link/components/cqrs/command_handler.go" first_line_contains="// CommandHandler" last_line_contains="type CommandHandler" padding_after="0" %}}
-{{% /render-md %}}
#### Command and Event Marshaler
-{{% render-md %}}
{{% load-snippet-partial file="src-link/components/cqrs/marshaler.go" first_line_contains="// CommandEventMarshaler" last_line_contains="NameFromMessage(" padding_after="1" %}}
-{{% /render-md %}}
## Usage
@@ -132,25 +105,19 @@ The domain is simple:
For the beginning, we need to simulate the guest's action.
-{{% render-md %}}
{{% load-snippet-partial file="src-link/_examples/basic/5-cqrs-protobuf/main.go" first_line_contains="bookRoomCmd := &BookRoom{" last_line_contains="panic(err)" padding_after="1" %}}
-{{% /render-md %}}
### Command handler
`BookRoomHandler` will handle our command.
-{{% render-md %}}
{{% load-snippet-partial file="src-link/_examples/basic/5-cqrs-protobuf/main.go" first_line_contains="// BookRoomHandler is a command handler" last_line_contains="// OrderBeerOnRoomBooked is a event handler" padding_after="0" %}}
-{{% /render-md %}}
### Event handler
As mentioned before, we want to order a beer every time when a room is booked (*"Whenever a Room is booked"* post-it). We do it by using the `OrderBeer` command.
-{{% render-md %}}
{{% load-snippet-partial file="src-link/_examples/basic/5-cqrs-protobuf/main.go" first_line_contains="// OrderBeerOnRoomBooked is a event handler" last_line_contains="// OrderBeerHandler is a command handler" padding_after="0" %}}
-{{% /render-md %}}
`OrderBeerHandler` is very similar to `BookRoomHandler`. The only difference is, that it sometimes returns an error when there are not enough beers, which causes redelivery of the command.
You can find the entire implementation in the [example source code](https://github.com/ThreeDotsLabs/watermill/tree/master/_examples/basic/5-cqrs-protobuf/?utm_source=cqrs_doc).
@@ -169,9 +136,7 @@ To use event groups, you need to set `GenerateHandlerGroupSubscribeTopic` and `G
After that, you can use `AddHandlersGroup` on [`EventProcessor`](#event-processor).
-{{% render-md %}}
{{% load-snippet-partial file="src-link/_examples/basic/5-cqrs-protobuf/main.go" first_line_contains="eventProcessor.AddHandlersGroup(" last_line_contains="if err != nil {" padding_after="0" %}}
-{{% /render-md %}}
Both `GenerateHandlerGroupSubscribeTopic` and `GroupSubscriberConstructor` receives information about group name in function arguments.
@@ -179,30 +144,20 @@ Both `GenerateHandlerGroupSubscribeTopic` and `GroupSubscriberConstructor` recei
Since Watermill v1.3 it's possible to use generic handlers for commands and events. It's useful when you have a lot of commands/events and you don't want to create a handler for each of them.
-{{% render-md %}}
{{% load-snippet-partial file="src-link/_examples/basic/5-cqrs-protobuf/main.go" first_line_contains="cqrs.NewGroupEventHandler" last_line_contains="})," padding_after="0" %}}
-{{% /render-md %}}
Under the hood, it creates EventHandler or CommandHandler implementation.
It's available for all kind of handlers.
-{{% render-md %}}
{{% load-snippet-partial file="src-link/components/cqrs/command_handler.go" first_line_contains="// NewCommandHandler" last_line_contains="func NewCommandHandler" padding_after="0" %}}
-{{% /render-md %}}
-{{% render-md %}}
{{% load-snippet-partial file="src-link/components/cqrs/event_handler.go" first_line_contains="// NewEventHandler" last_line_contains="func NewEventHandler" padding_after="0" %}}
-{{% /render-md %}}
-{{% render-md %}}
{{% load-snippet-partial file="src-link/components/cqrs/event_handler.go" first_line_contains="// NewGroupEventHandler" last_line_contains="func NewGroupEventHandler" padding_after="0" %}}
-{{% /render-md %}}
### Building a read model with the event handler
-{{% render-md %}}
{{% load-snippet-partial file="src-link/_examples/basic/5-cqrs-protobuf/main.go" first_line_contains="// BookingsFinancialReport is a read model" last_line_contains="func main() {" padding_after="0" %}}
-{{% /render-md %}}
### Wiring it up
@@ -215,9 +170,7 @@ It will also show you how to use some standard messaging patterns, like metrics,
Let's go back to the CQRS. As you already know, CQRS is built from multiple components, like Command or Event buses, handlers, processors, etc.
-{{% render-md %}}
{{% load-snippet-partial file="src-link/_examples/basic/5-cqrs-protobuf/main.go" first_line_contains="main() {" last_line_contains="err := router.Run(" padding_after="3" %}}
-{{% /render-md %}}
And that's all. We have a working CQRS application.
diff --git a/docs/content/docs/getting-started.md b/docs/content/docs/getting-started.md
index 4cebfc3e8..4c0765937 100644
--- a/docs/content/docs/getting-started.md
+++ b/docs/content/docs/getting-started.md
@@ -3,12 +3,10 @@ title = "Getting started"
description = "Watermill up and running"
weight = -9999
draft = false
-toc = true
bref = "Watermill up and running"
-type = "docs"
+++
-### What is Watermill?
+## What is Watermill?
Watermill is a Go library for working with message streams.
You can use it to build event-driven systems with popular Pub/Sub implementations like Kafka or RabbitMQ, as well as HTTP or Postgres if that fits your use case.
@@ -17,7 +15,7 @@ It comes with a set of Pub/Sub implementations and can be easily extended.
Watermill also ships with standard middlewares like instrumentation, poison queue, throttling, correlation,
and other tools used by every message-driven application.
-### Why use Watermill?
+## Why use Watermill?
When using microservices, synchronous communication is not always the right choice.
Asynchronous methods became a new standard way to communicate.
@@ -32,13 +30,13 @@ It provides all you need to build an application based on events or other asynch
**Watermill is NOT a framework**.
It's a lightweight library that's easy to plug in or remove from your project.
-### Install
+## Install
```bash
go get -u github.com/ThreeDotsLabs/watermill
```
-### One-Minute Background
+## One-Minute Background
The idea behind event-driven applications is always the same: listen to and react to incoming messages.
Watermill supports this behavior for multiple [publishers and subscribers]({{< ref "/pubsubs" >}}).
@@ -54,7 +52,9 @@ They build on top of each other, each step providing a higher-level API:
* The `Router` is similar to HTTP routers you probably know. It introduces message handlers.
* The `CQRS` component adds generic handlers without needing to marshal and unmarshal messages yourself.
-
+
+
+ 
-
-