Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/canonicalk8s/.custom_wordlist.txt
Original file line number Diff line number Diff line change
Expand Up @@ -530,6 +530,7 @@ ubuntu
Ubuntu Core
Ubuntu Pro
Ubuntu Server
UFW
uid
UIDs
unix
Expand Down
1 change: 1 addition & 0 deletions docs/canonicalk8s/snap/howto/networking/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,4 +15,5 @@ Use an alternative CNI <alternative-cni.md>
Enable Dual-Stack networking <dualstack.md>
Set up an IPv6-only cluster <ipv6.md>
Configure proxy settings <proxy.md>
Configure UFW <ufw.md>
```
180 changes: 180 additions & 0 deletions docs/canonicalk8s/snap/howto/networking/ufw.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,180 @@
# How to configure Uncomplicated Firewall (UFW)

In this how-to we present a set of firewall rules/guidelines
you should consider when setting up {{product}}.
Comment on lines +3 to +4
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
In this how-to we present a set of firewall rules/guidelines
you should consider when setting up {{product}}.
This how-to presents a set of firewall rules/guidelines
that should be considered when setting up {{product}}.

(let's use an active voice)

Be aware that these rules may be incompatible with your network setup
and we recommend you review and tune them to match your needs.
Comment on lines +5 to +6
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sounds like a note can we format it accordingly please?
So starting with: ```{note}

Comment on lines +5 to +6
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Be aware that these rules may be incompatible with your network setup
and we recommend you review and tune them to match your needs.
Please be aware that these rules may be incompatible with your network setup, so we recommend reviewing and adjusting them to match your needs.

(active voice + causal conjunction)


Also, be aware that for each service hosted in Kubernetes,
the firewall rules need to be reviewed as there might be
special requirements for each.
Comment on lines +8 to +10
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Also, be aware that for each service hosted in Kubernetes,
the firewall rules need to be reviewed as there might be
special requirements for each.
Also, please review the firewall rules for each service hosted in Kubernetes, as there might be
special requirements for each.



## Prerequisites

This guide assumes the following:

- A machine with Ubuntu where you have installed
or you plan to install {{product}}.
- You have root or sudo access to the machine.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should you already have a Canonical K8s cluster or is this done before the cluster is deployed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. Have a look at the updated version of this section.

Comment on lines +17 to +19
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- A machine with Ubuntu where you have installed
or you plan to install {{product}}.
- You have root or sudo access to the machine.
- An ubuntu machine where {{product}} is installed or will be installed.
- Root or sudo access to the machine.

(active voice)


## Install and enable UFW

Uncomplicated Firewall needs to be configured on all nodes of {{product}}.
To do so try:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
To do so try:
To do this, run:


```sh
sudo apt update
sudo apt install ufw
```

To verify UFW is installed try:

```sh
sudo ufw status verbose
```

If you need to maintain ssh access to the machine, make sure you configure
UFW to allow `OpenSSH` before enabling it:
Comment on lines +37 to +38
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
If you need to maintain ssh access to the machine, make sure you configure
UFW to allow `OpenSSH` before enabling it:
To maintain SSH access to the machine, allow `OpenSSH` through UFW before enabling the firewall:


```sh
sudo ufw allow OpenSSH
```

Now you are ready to enable UFW:

```sh
sudo ufw enable
```

## Allow forwarding

Forwarding is needed because containers typically live in isolated networks
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Forwarding is needed because containers typically live in isolated networks
Package forwarding is needed because containers typically live in isolated networks

and expect the host-to-route traffic between their internal network and the
outside world to be allowed.

First edit `/etc/default/ufw` and allow UFW to route/forward packets:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
First edit `/etc/default/ufw` and allow UFW to route/forward packets:
First, edit `/etc/default/ufw` and allow UFW to route/forward packets:


```sh
DEFAULT_FORWARD_POLICY="ACCEPT"
```

Enable IP forwarding by editing `/etc/sysctl.conf` so it persists through
system reboots:

```sh
net.ipv4.ip_forward=1
```

Or use `sysctl` directly so forwarding is applied immediately,
without rebooting the system:

Comment on lines +61 to +71
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Enable IP forwarding by editing `/etc/sysctl.conf` so it persists through
system reboots:
```sh
net.ipv4.ip_forward=1
```
Or use `sysctl` directly so forwarding is applied immediately,
without rebooting the system:
If you want the forwarding rules to persist through system reboots, enable IP forwarding by editing `/etc/sysctl.conf`:
```sh
net.ipv4.ip_forward=1

Otherwise, use sysctl directly to apply the forwarding rules immediately
without rebooting the system:

(Let's make the two options more clearer in the wording)

```sh
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you edit the /etc/sysctl then you don't need to run this command correct? I think this is slightly unclear

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've clarified the need for this command in the revised version of this PR.

sudo sysctl -w net.ipv4.ip_forward=1
```

Reload UFW:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Reload UFW:
Reload UFW to apply the changes:


```sh
sudo ufw reload
```

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could optionally add a verification step here using ufw status. Wdyt?

## Allow access to the Kubernetes services

Services such as CoreDNS require access to the Kubernetes API
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could add a link to CoreDNS docs here.

server listening on port 6443.

Allow traffic on port 6443 with:

``` sh
sudo ufw allow 6443/tcp
```

Services such as the metrics-server need access to the kubelet,
controller manager and kube scheduler to query for metrics.

Kubelet runs on all nodes, so allow traffic on port 10250 on all nodes:

```sh
sudo ufw allow 10250/tcp
```

The kube-controller-manager and kube-scheduler run only on
the control plane so allow traffic on ports 10257 and 10259
on control plane nodes:
Comment on lines +102 to +104
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The kube-controller-manager and kube-scheduler run only on
the control plane so allow traffic on ports 10257 and 10259
on control plane nodes:
The kube-controller-manager and kube-scheduler only run on
the control plane, therefore permit traffic on ports 10257 and 10259
of the control plane nodes:


```sh
sudo ufw allow 10257/tcp
sudo ufw allow 10259/tcp
```

## Allow cluster formation

To form a High Availability (HA) cluster the datastore used by Kubernetes
(dqlite/etcd) needs to establish a direct connection among its peers.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nhennigan We capitalize Dqlite in the rest of the docs but keep etcd lower-case. I think we should be uniform in out capitalization with etcd/dqlite. Wdyt?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Yes Dqlite should be capitalized according to their docs. I think the confusion stems from the fact on github it is listed with a lower case. etcd is the name of the product and they do not capitalize

In dqlite this is done through port 9000 while on etcd port 2380 is used.

Allow traffic on port 9000 on control plane nodes with dqlite:

```sh
sudo ufw allow 9000/tcp
```

Allow traffic on port 2380 on control plane nodes with etcd:

```sh
sudo ufw allow 2380/tcp
```
Comment on lines +123 to +127
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about port 2379 for client communication?


Cluster formation is overseen by a Kubernetes daemon running on all nodes
on port 6400.

Allow traffic on port 6400:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Allow traffic on port 6400:
Open port 6400 to permit cluster formation traffic:


```sh
sudo ufw allow 6400/tcp
```

## Allow CNI specific communication
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
## Allow CNI specific communication
## Enable CNI specific communication

(we've repeated allow a lot)


If you are using the default network plugin (Cilium),
you should consider the following firewall rules.
Comment on lines +140 to +141
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
If you are using the default network plugin (Cilium),
you should consider the following firewall rules.
When using the default network plugin (Cilium),
consider the following firewall rules.


Allow cluster-wide Cilium agent health checks and VXLAN traffic:

```sh
sudo ufw allow 4240/tcp
sudo ufw allow 8472/udp
```

## UFW troubleshooting

The [ports-and-services] page has a list of all ports {{product}} uses.

To inspect a failing service you can enable logging:

```sh
sudo ufw logging on
```

Monitor the firewall logs with:

```sh
tail -f /var/log/ufw.log
```
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What can you learn form these logs? Which ports are trying to be accessed? Maybe we should add a comment here along the lines of "you should be able to see where the traffic is being dropped" or "this may help you identify any other ports or services you need to enable with UTW"


The logs will show you which packets are dropped, their destination
and source as well as the protocol used and the destination port.
This information may help you identify any other ports or services
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This information may help you identify any other ports or services
This information helps to identify any other ports or services

you need to enable within UFW.

To keep the resources used by UFW to a minimum you can disable logging:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
To keep the resources used by UFW to a minimum you can disable logging:
After troubleshooting, keep the resources used by UFW to a minimum by disabling logging again:


```sh
sudo ufw logging off
```


<!-- LINKS -->

[ports-and-services]: ../../reference/ports-and-services
Loading