Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ci/do_ci.sh
Original file line number Diff line number Diff line change
Expand Up @@ -960,7 +960,7 @@ case $CI_TARGET in
;;

verify_examples)
run_ci_verify "*" "win32-front-proxy|shared"
run_ci_verify "*" "win32-front-proxy|shared|dlb"
;;

verify.trigger)
Expand Down
6 changes: 3 additions & 3 deletions docs/root/configuration/other_features/dlb.rst
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ If enabled, the DLB connection balancer will:
When new connections come, one worker thread will accept it and send it to DLB hardware. DLB hardware
does balancing then trigger one worker thread to receive via libevent.

Installing DLB
--------------
Using DLB
---------

For information on how to build/install and use libdlb see `the getting started guide <https://downloadmirror.intel.com/727424/DLB_Driver_User_Guide.pdf>`_.
You can see full example in this [doc](https://www.envoyproxy.io/docs/envoy/latest/start/sandboxes/dlb.html).
3 changes: 3 additions & 0 deletions examples/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ filegroup(
"jaeger-native-tracing/*",
"opentelemetry/otel-collector-config.yaml",
"**/*docker-compose*.yaml",
# no available devices in the CI env
"dlb/*.yaml",
# Contrib extensions tested over in contrib.
"golang-http/*.yaml",
"golang-network/*.yaml",
Expand All @@ -43,6 +45,7 @@ filegroup(
"kafka/*.yaml",
],
exclude = [
"dlb/*.yaml",
"**/*docker-compose*.yaml",
],
),
Expand Down
45 changes: 45 additions & 0 deletions examples/dlb/dlb_example_config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
static_resources:
listeners:
- address:
socket_address:
address: 0.0.0.0
port_value: 10000
connection_balance_config:
extend_balance:
name: envoy.network.connection_balance.dlb
typed_config:
"@type": type.googleapis.com/envoy.extensions.network.connection_balance.dlb.v3alpha.Dlb
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
codec_type: AUTO
stat_prefix: ingress_http
route_config:
name: local_route
virtual_hosts:
- name: backend
domains:
- "*"
routes:
- match:
prefix: "/"
route:
cluster: service1
http_filters:
- name: envoy.filters.http.router
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router

clusters:
- name: service1
load_assignment:
cluster_name: service1
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: 127.0.0.1
port_value: 12000
105 changes: 105 additions & 0 deletions examples/dlb/example.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
.. _install_sandboxes_dlb:

DLB
===

By enabling connection balancer in Envoy you can balance the connections across the threads and improve performance.

This sandbox provides an example about how to enable DLB connection balanace.

.. note::
Please run below command to check your CPU supports DLB:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yanavlasov I put command here to help users detect its CPU can support DLB.


.. code-block:: console
$ lspci -d :2710

You should get output like below:

.. code-block:: text
5a:00.0 Co-processor: Intel Corporation Device 2710
6b:00.0 Co-processor: Intel Corporation Device 2710
7c:00.0 Co-processor: Intel Corporation Device 2710
8d:00.0 Co-processor: Intel Corporation Device 2710
The output that is not empty means CPU supports DLB. The number and PCIE address of DLB devices vary from CPU to CPU.

The minimum support kernel version is 5.15.

Step 1: Install DLB Driver
**************************

You can download the DLB driver release tarball from the `DLB website <https://www.intel.com/content/www/us/en/download/686372/intel-dynamic-load-balancer.html>`_.

To install it refer to `the getting started guide <https://downloadmirror.intel.com/727424/DLB_Driver_User_Guide.pdf>`_.

Step 2: Run Envoy with DLB connection balanace enabled
******************************************************

With the example configuration Envoy listens on port 10000 and proxies to an upstream server listening on port 12000.

.. literalinclude:: _include/dlb/dlb_example_config.yaml
:language: yaml
:lines: 7-11
:lineno-start: 7
:linenos:
:caption: :download:`dlb_example_config.yaml <_include/dlb/dlb_example_config.yaml>`

.. code-block:: console
$ ./envoy --concurrency 2 -c dlb_example_config.yaml --log-level debug &> envoy-dlb.log
After Envoy starts, you should see logs similar to:

.. code-block:: console
$ grep dlb envoy-dlb.log
.. code-block:: text
[2024-07-08 10:05:00.113][3596312][info][main] [source/server/server.cc:434] envoy.network.connection_balance: envoy.network.connection_balance.dlb
[2024-07-08 10:05:00.241][3596312][debug][config] [contrib/dlb/source/connection_balancer_impl.cc:92] dlb available resources: domains: 32, LDB queues: 32, LDB ports: 64, ES entries: 2048, Contig ES entries: 2048, LDB credits: 8192, Config LDB credits: 8192, LDB credit pools: 64
Step 3: Run the upstream service
********************************

.. code-block:: console
$ docker run -d -p 12000:80 nginx
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the sandboxes use compose to start and connect services - this should not be any different

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know that, but compose is not good at using hardware device in the container, also map the hardware introduce extra knowledge and difficulty for beginners, so here I use docker directly for all actions.

Step 3: Test
************

Visit the upstream service by Envoy endpoint:

.. code-block:: console
$ curl localhost:10000 | grep Welcome
You should get output from Nginx:

.. code-block:: text
<title>Welcome to nginx!</title>
Check the log, you should see the latest contents similar to:

.. code-block:: text
[2024-07-08 10:10:48.099][3598062][debug][connection] [contrib/dlb/source/connection_balancer_impl.cc:283] worker_1 dlb send fd 49
[2024-07-08 10:10:48.099][3598066][debug][connection] [contrib/dlb/source/connection_balancer_impl.cc:300] worker_3 get dlb event 1
[2024-07-08 10:10:48.099][3598066][debug][connection] [contrib/dlb/source/connection_balancer_impl.cc:317] worker_3 dlb recv 49
[2024-07-08 10:10:48.099][3598066][debug][connection] [contrib/dlb/source/connection_balancer_impl.cc:297] worker_3 dlb receive none, skip
Above logs show that DLB balanaces a connection from worker 1 to worker 3.

.. seealso::
:ref:`DLB connection balanace API <envoy_v3_api_msg_extensions.network.connection_balance.dlb.v3alpha.Dlb>`
API and configuration reference for Envoy's DLB connection balanace.

:ref:`Connection balance configuration <envoy_v3_api_field_config.listener.v3.Listener.connection_balance_config>`
Configuration referenc for Envoy's connection balanace.

`DLB <https://networkbuilders.intel.com/solutionslibrary/queue-management-and-load-balancing-on-intel-architecture>`_
The Intel DLB website.