Skip to content

Commit

Permalink
Merge remote-tracking branch 'origin/master'
Browse files Browse the repository at this point in the history
# Conflicts:
#	tools/mill/go.mod
#	tools/mill/go.sum
  • Loading branch information
roblaszczak committed Oct 9, 2021
2 parents 188268e + 650d0ad commit 08b7014
Show file tree
Hide file tree
Showing 12 changed files with 294 additions and 18 deletions.
4 changes: 2 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## How can I help?

We are always happy to help you in contributing to Watermill. If you have any ideas, please let us know on our [Slack channel](https://github.com/ThreeDotsLabs/watermill#support).
We are always happy to help you in contributing to Watermill. If you have any ideas, please let us know on our [Discord server](https://watermill.io/support/).

There are multiple ways in which you can help us.

Expand All @@ -25,7 +25,7 @@ When adding a new Pub/Sub implementation, you should start with this guide: [htt
### New ideas

If you have any idea that is not covered in the issues list, please post a new issue describing it.
It's recommended to discuss your idea on Slack/GitHub before creating production-ready implementation - in some situations, it may save a lot of your time before implementing something that can be simplified or done more easily. :)
It's recommended to discuss your idea on [Discord](https://discord.gg/QV6VFg4YQE)/GitHub before creating production-ready implementation - in some situations, it may save a lot of your time before implementing something that can be simplified or done more easily. :)

In general, it's helpful to discuss a Proof of Concept to align with the idea.

Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,7 @@ Here's the short version for message size of 16 bytes.

If you didn't find the answer to your question in [the documentation](https://watermill.io/), feel free to ask us directly!

Please join us on the `#watermill` channel on the [Gophers slack](https://gophers.slack.com/): You can get an invite [here](https://gophersinvite.herokuapp.com/).
Please join us on the `#watermill` channel on the [Three Dots Labs Discord](https://discord.gg/QV6VFg4YQE).

Every bit of feedback is very welcome and appreciated. Please submit it using [the survey](https://www.surveymonkey.com/r/WZXD392).

Expand Down
1 change: 1 addition & 0 deletions docs/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ cloneOrPull "https://github.com/ThreeDotsLabs/watermill-io.git" content/src-link
cloneOrPull "https://github.com/ThreeDotsLabs/watermill-kafka.git" content/src-link/watermill-kafka
cloneOrPull "https://github.com/ThreeDotsLabs/watermill-nats.git" content/src-link/watermill-nats
cloneOrPull "https://github.com/ThreeDotsLabs/watermill-sql.git" content/src-link/watermill-sql
cloneOrPull "https://github.com/ThreeDotsLabs/watermill-firestore.git" content/src-link/watermill-firestore


python3 ./extract_middleware_godocs.py > content/src-link/middleware-defs.md
Expand Down
4 changes: 4 additions & 0 deletions docs/config.toml
Original file line number Diff line number Diff line change
Expand Up @@ -38,3 +38,7 @@ googleAnalytics = "UA-128588911-2"
name = "Support"
weight = 0
url = "/support/"

[markup]
defaultMarkdownHandler = 'blackfriday'

84 changes: 84 additions & 0 deletions docs/content/pubsubs/firestore.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
+++
title = "Firestore Pub/Sub"
description = "A scalable document database from Google"
date = 2021-07-29T15:30:00+02:00
bref = "A scalable document database from Google"
weight = 0
type = "docs"
toc = false
+++

### Firestore Pub/Sub

Cloud Firestore is a cloud-hosted, NoSQL database from Google.

This Pub/Sub comes with two publishers. To publish messages in a transaction
use the `TransactionalPublisher`. If you do not want to publish messages in
transaction use the normal `Publisher`.

Using Firestore as a Pub/Sub instead of using a dedicated Pub/Sub system can be
useful to publish messages in transaction while at the same time saving other
data in Firestore. Thanks to that the data and the messages can be consistently
persisted. If the messages and the data weren't being published transactionally
you could end up in situations where messages were emitted even though the data
wasn't saved or messages weren't emitted even though the data was saved. After
transactionally publishing messages in Firestore you can then subscribe to them
and relay them to a different Pub/Sub system.

Godoc: <https://pkg.go.dev/github.com/ThreeDotsLabs/watermill-firestore>

Firestore documentation: <https://firebase.google.com/docs/firestore/>

#### Characteristics

| Feature | Implements | Note |
| ------- | ---------- | ---- |
| ConsumerGroups | no | |
| ExactlyOnceDelivery | no | |
| GuaranteedOrder | no | |
| Persistent | yes | |

#### Configuration

##### Publisher configuration

{{% render-md %}}
{{% load-snippet-partial file="src-link/watermill-firestore/pkg/firestore/publisher.go" first_line_contains="type PublisherConfig struct {" last_line_equals="}" %}}
{{% /render-md %}}

##### Subscriber configuration

{{% render-md %}}
{{% load-snippet-partial file="src-link/watermill-firestore/pkg/firestore/subscriber.go" first_line_contains="type SubscriberConfig struct {" last_line_equals="}" %}}
{{% /render-md %}}

##### Subscription name

To receive messages published to a topic, you must create a subscription to
that topic. Only messages published to the topic after the subscription is
created will be received by the subscribers.

A topic can have multiple subscriptions, but a given subscription belongs to a
single topic.

In Watermill, the subscription is created automatically during calling
`Subscribe()`. Subscription name is generated by function passed to
`SubscriberConfig.GenerateSubscriptionName`. By default, it is just the topic
name with a suffix `_sub` appended to it.

If you want to consume messages from a topic with multiple subscribers
processing the incoming messages in a different way, you should use a custom
function to generate unique subscription names for each subscriber.

#### Marshaler

Watermill's messages cannot be stored directly in Firestore. The marshaler is
responsible for converting them to a type which can be stored by Firestore.
The default implementation should be enough for most applications so it is
unlikely that you need to implement your own marshaler.

{{% render-md %}}
{{% load-snippet-partial file="src-link/watermill-firestore/pkg/firestore/marshaler.go" first_line_contains="// Marshaler" last_line_equals="}" padding_after="0" %}}
{{% /render-md %}}


2 changes: 1 addition & 1 deletion docs/content/support.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ description = ""

### Community Support

Please join us on the `#watermill` channel on the [Gophers slack](https://gophers.slack.com/). You can get the invite [here](https://gophersinvite.herokuapp.com/).
Please join us on the `#watermill` channel on the [Three Dots Labs discord](https://discord.gg/QV6VFg4YQE).

### Professional Support

Expand Down
7 changes: 6 additions & 1 deletion docs/layouts/shortcodes/load-snippet-partial.html
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@

{{ $first_line_contains := (.Get "first_line_contains") }}
{{ $last_line_contains := (.Get "last_line_contains") }}
{{ $last_line_equals := (.Get "last_line_equals") }}

{{ $show_line := false }}

Expand Down Expand Up @@ -51,6 +52,10 @@
{{ $last_line_found = true }}
{{ end }}

{{ if and ($first_line_found) (eq $elem_val $last_line_equals) (ne $last_line_equals "") }}
{{ $last_line_found = true }}
{{ end }}

{{ if and $last_line_found $show_line }}
{{ if gt $padding_after 0 }}
{{ $padding_after = sub $padding_after 1}}
Expand All @@ -72,4 +77,4 @@

{{if and (not $last_line_found) (ne $last_line_contains "") }}
{{ errorf "`last_line_contains` %s not found in %s snippet" $last_line_contains $file }}
{{end}}
{{end}}
4 changes: 2 additions & 2 deletions message/router.go
Original file line number Diff line number Diff line change
Expand Up @@ -279,12 +279,12 @@ func (r *Router) AddNoPublisherHandler(
subscribeTopic string,
subscriber Subscriber,
handlerFunc NoPublishHandlerFunc,
) {
) *Handler {
handlerFuncAdapter := func(msg *Message) ([]*Message, error) {
return nil, handlerFunc(msg)
}

r.AddHandler(handlerName, subscribeTopic, subscriber, "", disabledPublisher{}, handlerFuncAdapter)
return r.AddHandler(handlerName, subscribeTopic, subscriber, "", disabledPublisher{}, handlerFuncAdapter)
}

// Run runs all plugins and handlers and starts subscribing to provided topics.
Expand Down
6 changes: 3 additions & 3 deletions netlify.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,14 @@
publish = "docs/public/"

[context.production.environment]
HUGO_VERSION = "0.53"
HUGO_VERSION = "0.87.0"
HUGO_ENV = "production"
HUGO_ENABLEGITINFO = "true"

[context.deploy-preview.environment]
HUGO_VERSION = "0.53"
HUGO_VERSION = "0.87.0"
HUGO_ENABLEGITINFO = "true"

[context.branch-deploy.environment]
HUGO_VERSION = "0.53"
HUGO_VERSION = "0.87.0"
HUGO_ENABLEGITINFO = "true"
10 changes: 10 additions & 0 deletions tools/mill/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,3 +88,13 @@ mill googlecloud subscription rm <subscription_id>
```

Additional flags are available for `subscription add` to regulate the newly created subscription's settings.

#### Listing subscriptions

You can use `mill` to list existings subscriptions:

```bash
mill googlecloud subscription ls [-t topic]
```

The topic is optional. If omitted, all topics will be listed with their subscriptions.
Loading

0 comments on commit 08b7014

Please sign in to comment.