Design

This is a quick attempt to define ExaBGP's internal .. This document should improve over time.

ExaBGP run a single loop, the loop is managed by the reactor which will share time between the network and the management of the forked process.

As the code needs to be non blocking, extensive use of a feature called co-routine is used. A python co-routine is a resumable function, instead of returning a value a function yield it and at the next call the execution resumes from the yield point. Those function when run create a generator, which is an iterator object which will when requested, by using the funtion next() on it, the next value. The good thing is that is is possible to generate and consume it (like for unix pipes) without using massive among of memory.

The reactor and networking code use coroutine. The networking code is split in three level, connection, protocol, and peer :

• connection provide using co-routing a non block IO layer this is a forever running generator which will return nothing when nothing is there, or data when data is found, it uses unix select(2) on a non-blocking socket.
• protocol wraps connection to provide a BGP message level api, it returns a NOP message when no new messages are available or a Message subclass when new messages can be fully formed ( Open, Update, KeepAlive, etc..)
• each peer gets one instance of Peer which then perform the protocol discussion, again using a co-routing.
• the reactor then manage all the peers and the Process forked and dispatch message between them.

The whole code in peer is wrapped in a try / except block. On error an exception is raised and will be intercepted by that block which will send a notification to the other side, tear down the session and re-establish it if required.

The forked process are able to generate a very large among of data so in order to be fair the reactor runs cycle of one second, which guarantee that each second KeepAlive work (heartbeat between routers) is performed. Up to 0.5 seconds, the reactor consumes data from the PIPE, for the rest of the second, ExaBGP handles the networking I/O, should the networking work not take up to a second more data is consumed from the PIPE and should not data be there, ExaBGP will then sleep to the end of the second.

When routes to be announced to the peer are generated by the PIPE, the data is parsed and an single NLRI with its attributes, callled a Change, is generated. This change is set added by the reactor to the outgoing RIB of the peers needing to announce it. The Changes are grouped by attributes, so the Update message created can network efficient. When the reactor comes to handle each peer, each RIB generate a co-routine which is consumed by the peer networking code, obviously the consumption code is a co-routine itself managed by the peer so this creation is non-blocking too.