Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reverse connection: reverse connection bootstrap extension #37819

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Reverse connections bootstrap extension

2429282
Select commit
Loading
Failed to load commit list.
Open

reverse connection: reverse connection bootstrap extension #37819

Reverse connections bootstrap extension
2429282
Select commit
Loading
Failed to load commit list.
CI (Envoy) / Mobile/Perf skipped Dec 26, 2024 in 0s

Check was skipped

This check was not triggered in this CI run

Details

Request (pr/37819/main@2429282)

basundhara-c @basundhara-c 2429282 #37819 merge main@f122866

reverse connection: reverse connection bootstrap extension

Commit Message: The "envoy.bootstrap.reverse_connection" bootstrap extension is the central part of the reverse connection workflow and brings up several reverse connection entities described in this github issue. This PR is strictly tied with #37368.

Additional Description: The following reverse connection entities are relevant on the initiator envoy side:
A ReverseConnRegistry singleton is defined as part of the bootstrap extension which holds a TLS slot. The TLS slot holds a functor that creates a RCThreadLocalRegistry, which wraps over all the thread local entities that need to be initialized for reverse connections, in particular:

  • The ReverseConnectionInitiator is a thread local entity created unique for each Listener Tag. On being created, the RCInitiator initiates reverse connections to clusters based on metadata present in the listener config. Upon connection closure, the owning RCInitiator is invoked to re-initiate connections.
  • The ReverseConnectionManager is a a single thread local resource that manages the lifecycle of all ReverseConnectionInitiators. This is initialized as part of the RCThreadLocalRegistry and stores information about ReverseConnectionInitiator and the connections owned by them, and is called upon to invoke RCInitiators and replenish connections in case of closure.

The following entities are relevent on the responder envoy side:

  • The ReverseConnectionHandler is a thread local socket manager that, on the responder envoy side, stores mappings of initiator node/cluster information with the corresponding reverse connection sockets.
  • When a request arrives that needs to be sent over a reverse connection, the allocateConnPool API invokes the RCHandler to pick a cached socket for the host to sent the request.
  • The Conn Pool then creates a ReversedClientConnectionImpl over the socket. The ReversedClientConnectionImpl skips connecting to the upstream host (because a socket is available) and calls the RCHandler to cleanup in case of socket closure. Note that re-initiaiton in case of closure is done only on the initiator envoy.

Risk Level: Low
Testing:
Docs Changes:
Release Notes:
Platform Specific Features:
[Optional Runtime guard:]
[Optional Fixes #Issue]
[Optional Fixes commit #PR or SHA]
[Optional Deprecated:]
[Optional API Considerations:]

Environment

Request variables

Key Value
ref a8ee0d2
sha 2429282
pr 37819
base-sha f122866
actor basundhara-c @basundhara-c
message reverse connection: reverse connection bootstrap extension...
started 1735252347.796587
target-branch main
trusted false
Build image

Container image/s (as used in this CI run)

Key Value
default envoyproxy/envoy-build-ubuntu:d2be0c198feda0c607fa33209da01bf737ef373f
mobile envoyproxy/envoy-build-ubuntu:mobile-d2be0c198feda0c607fa33209da01bf737ef373f
Version

Envoy version (as used in this CI run)

Key Value
major 1
minor 33
patch 0
dev true