Skip to content

Latest commit

 

History

History
1882 lines (1580 loc) · 80.3 KB

BAYEUX.adoc

File metadata and controls

1882 lines (1580 loc) · 80.3 KB

SmallMind Bayeux: The Oumuamua Server

Web clients need to be told about things that happen on the server: a chat message, a price tick, a build status change, a row that another user just edited. The request/response shape of HTTP inverts that need — it assumes the client always speaks first. Application teams have been routing around that mismatch for twenty years with long polling, Server-Sent Events, custom WebSocket framing, and a scatter of ad-hoc JSON conventions on top of each.

Bayeux is the protocol that standardises the routing-around. It defines a small set of meta channels (/meta/handshake, /meta/connect, /meta/subscribe, /meta/disconnect, …), a publish/subscribe contract on user-defined channels (/chat/room/42, /pricing/AAPL), and a transport negotiation step that picks whichever of long polling or WebSocket both sides support. A Bayeux client starts with a handshake, subscribes to some channels, then simply receives messages as the server produces them — whether the transport underneath is a held HTTP connection or an open WebSocket.

CometD is the canonical Java Bayeux implementation. It works, but it is an ecosystem: a dozen artifacts, a servlet bridge, a separate WebSocket bridge, and an annotation-driven service layer whose bean graph is not cheap to reason about. If the only thing an application actually needs is "accept Bayeux clients, route messages between them, maybe reach another node over Kafka, and do that from inside my own servlet container," CometD has a considerably larger surface area than the requirement.

Oumuamua is the SmallMind Bayeux server — a compact, strongly typed, protocol-neutral implementation with exactly three modules, no annotation magic, a clean API/SPI/implementation split, and a channel tree that the application can inspect and modify at runtime. It speaks long polling and WebSocket, supports at-least-once delivery through the standard ack extension, and it is wired with Spring XML (or plain Java) into whatever servlet container the application already runs.

Concretely, the reasons an application team picks Oumuamua over the alternative might include:

Modular clustering over enterprise messaging infrastructure

Clustering is not bolted onto the server; it is an explicit Backbone SPI that any inter-node messaging bus can implement. The repository ships a Kafka implementation (KafkaBackbone) because Kafka is the bus most deployments already have, but an application that runs on JMS, Redis Streams, NATS, or a cloud-native pub/sub can plug its own Backbone in without touching the rest of the server. Fan-out semantics (who sees what, when loopback is filtered) are the same regardless of the transport underneath.

A much more linear code path than CometD

Reading a packet from receipt to delivery in Oumuamua is a short straight line: servlet or WebSocket endpoint → codec → Server.onRequest → meta / service / publish dispatch → Server.deliver → per-session queue or stream. There is no annotation scanner, no registry of dynamically discovered services, and no reflective bean graph standing between the entry point and the wire. When something goes wrong, the stack trace is the flow chart.

First-class support for high-volume streaming channels

A channel can be flagged streaming, either individually or via a pattern in OumuamuaConfiguration.setStreamingPaths. Streaming channels dispatch each packet immediately rather than accumulating in the long-poll queue, and the AckExtension steps aside on transports that do not require it, so tickers, telemetry feeds, and similar high-rate channels do not pay the ack bookkeeping cost they would otherwise. overflowLogLevel and maxLongPollQueueSize are tunable per deployment so operators can make the back-pressure trade-off explicitly.

Hooks to extend or rewrite parts of the server itself

Every major contract the server depends on is an interface or a minimal abstract class: Codec, Value, Protocol, Transport, SecurityPolicy, Backbone, BayeuxService, ChannelInitializer, and the four Server.Listener subtypes. A team that needs to rewrite a section of the server — say, to add a new transport, an unusual authentication model, or a different serialization format — subclasses the SPI type and wires it in, without forking the running server.

A standard internal message representation, and a way to change it

Inside the server, a Bayeux message is an ObjectValue<V> from the Value<V> hierarchy, which both the codec and the listeners know how to read and rewrite. The shipped OrthodoxValue / OrthodoxCodec is the default, but V is a type parameter all the way through, so an application that needs a different in-memory representation — pooled buffers, a zero-copy document tree, a Kafka-record-compatible shape — supplies its own Value family and Codec without touching OumuamuaServer or the transports.

The name is deliberate. Oumuamua was an interstellar object that briefly passed through the solar system — small, fast-moving, and demonstrably not from around here. The server is named in the same spirit: it arrives in your application, does its job, and does not try to reshape the house around it.

Your question Where to look

I just want to send messages from the server to browsers.

Read Conceptual Model, then Installation, then Quick Start.

What exactly is a Bayeux channel, a session, and a packet in this code?

Conceptual Model.

I want to know every class in the API surface before I adopt this.

API Reference.

How do I plug in authentication, per-channel access control, or message auditing?

Extensibility.

How do I make this work across more than one application node?

Clustering With Kafka.

How do I write a server-side RPC handler that a Bayeux client can call?

Writing a Bayeux Service.

Something is wrong. A client never gets a message, or a session won’t handshake, or the server logs nothing.

Troubleshooting.

A few ideas recur throughout the server, so it is worth naming them before showing code.

Bayeux

The wire protocol. A JSON-framed publish/subscribe conversation that starts with a handshake on /meta/handshake and proceeds through /meta/connect, /meta/subscribe, and user channels. Every message is a JSON object with a channel field and a handful of optional fields (id, clientId, data, ext, advice, successful, error, subscription). The protocol does not dictate the transport; the handshake negotiates one.

Session

A connected Bayeux client, server-side. Has a unique id, a SessionState that walks INITIALIZED → HANDSHOOK → CONNECTED → DISCONNECTED exactly once, and a long-poll queue of pending outbound packets. The Session interface is also Attributed, so extensions hang their per-client state off it without introducing new fields.

Channel

A named delivery path with a set of subscribed sessions. Paths are slash-delimited. Three prefixes are reserved: /meta/… for protocol control channels, /service/… for server-side RPC handlers, and anything else for user-visible broadcast. A channel can be marked persistent (kept alive when empty), reflecting (messages echo back to their sender), and streaming (delivered immediately rather than batched).

Route

The parsed form of a channel path. A Route knows its segments, whether it ends in (single-level wild) or * (deep wild), and whether it is meta, service, or deliverable. Channels, services, and listeners all key off routes rather than raw strings.

Packet

An immutable envelope of one or more Bayeux Message`s aimed at a single route, tagged by `PacketType: REQUEST (client to server), RESPONSE (server reply to a request), or DELIVERY (fan-out to subscribers). Listeners operate on packets, not raw messages, so a listener can replace the entire batch in one step.

Message freezing

When a DELIVERY packet is handed to a channel, the carried messages are frozen — made immutable for the purposes of the current fan-out — before any Channel.PacketListener or Session.PacketListener can inspect them. This guarantees that if a session-level listener rewrites a field to personalise the packet for its own client, no other subscriber in the same fan-out sees that mutation. Personalisation stays local to one delivery stream.

Protocol vs. Transport

A Protocol is a named variant (servlet for HTTP/long polling, websocket for JSR-356 WebSockets). It owns one or more Transport instances, which are the actual wire implementations. Handshake negotiation selects the strongest supported protocol; the session’s isLongPolling() flag records what was chosen.

Backbone

The inter-node message distribution layer. A single-node deployment can omit it. A multi-node deployment supplies a Backbone implementation — in this repository that is KafkaBackbone — and every deliver with local=true fans the packet out to peer nodes over the bus.

Codec and Value

Bayeux messages are JSON; Codec<V> creates and parses them, and Value<V> is the parametrised JSON-value abstraction so that message payloads can be carried without committing to a single JSON library at the API boundary. The shipped OrthodoxCodec builds on Jackson for deserialization and uses its own OrthodoxValue hierarchy for the in-memory representation.

Service channel

A channel beginning with /service/. It is never broadcast — the server hands incoming messages to a registered BayeuxService, which computes a response packet and sends it back to the caller only. A client subscription to a service channel is acknowledged but never recorded, so nothing is ever delivered to it. This is how a browser-to-server RPC looks in Bayeux.

Ack extension

The standard ack protocol extension guarantees at-least-once delivery over long polling. The server attaches an incrementing id to each delivered packet; the client echoes the most recent id it has received on the next connect; the server discards everything up to and including that id and re-sends the rest. AckExtension implements the server side.

The module is split so that a library consumer imports only what it needs.

Artifact Purpose When you depend on it Module name

oumuamua-server-api

Contracts only: Server, Session, Channel, Packet, Route, Message, Protocol, Transport, SecurityPolicy, BayeuxService. No implementation, no Spring, no Kafka.

Shared library code that must describe Bayeux shapes (BayeuxService implementations, message handlers, custom Value types) without pulling in the server.

org.smallmind.bayeux.oumuamua.server.api

oumuamua-server-spi

Service-provider base classes: AbstractProtocol, AbstractAttributed, DefaultRoute, OpenSecurityPolicy, the OrthodoxCodec / OrthodoxValue family, the Kafka Backbone, the AckExtension and TimesyncExtension, and the JSR-356 WebSocket protocol / transport.

Anyone writing a custom codec, a custom protocol, an additional extension, a new backbone, or any component that needs to subclass a server-provided base class.

org.smallmind.bayeux.oumuamua.server.spi

oumuamua-server-impl

The running server: OumuamuaServer, OumuamuaSession, OumuamuaChannel, the channel tree, OumuamuaServlet, WebSocketEndpoint, the long-polling servlet protocol, and OumuamuaConfiguration.

The application that actually hosts the Bayeux server.

org.smallmind.bayeux.oumuamua.server.impl

A consuming library that only needs to type Bayeux shapes depends on oumuamua-server-api only. An application that runs the server depends on oumuamua-server-impl (which pulls in the SPI and API transitively).

oumuamua-server-api ships an explicit module descriptor that exports its three packages (org.smallmind.bayeux.oumuamua.server.api, .backbone, and .json) and re-exports jakarta.servlet and org.smallmind.nutsnbolts (requires transitive), whose types appear in the API’s signatures. A module that requires org.smallmind.bayeux.oumuamua.server.api reads both without naming them. oumuamua-server-spi ships an explicit module descriptor that exports all ten of its packages and re-exports org.smallmind.bayeux.oumuamua.server.api, jakarta.servlet, org.smallmind.nutsnbolts, and org.smallmind.scribe.pen. Its eight optional dependencies are requires static, matching their optional POM dependencies, so each feature below works on a module path only when the application requires the listed module itself (or adds it with --add-modules):

Feature (package) Module the application requires

OrthodoxCodec and the Jackson deserializers (spi.json.orthodox, spi.json.jackson)

org.smallmind.web.json.scaffold, which re-exports tools.jackson.core and tools.jackson.databind

KafkaBackbone (spi.backbone.kafka)

org.smallmind.kafka.utility (its constructor takes KafkaServer[] and KafkaGroupProtocol), plus kafka.clients if the application uses Kafka types itself. kafka.clients is derived from the jar’s file name.

JSR-356 WebSocket protocol (spi.websocket.jsr356)

jakarta.websocket and jakarta.websocket.client (WebsocketConfiguration and WebsocketConfigurator expose Endpoint, Extension, and ServerEndpointConfig)

Latency metrics (spi.latency)

org.smallmind.claxon.registry

Warning

A jar on the module path is not enough. A module that requires only org.smallmind.bayeux.oumuamua.server.spi and builds new OrthodoxCodec(new JaxbDeserializer<>()) compiles, then fails at run time with NoClassDefFoundError: org/smallmind/web/json/scaffold/util/JsonCodec even though web-json-scaffold is on the module path; adding requires org.smallmind.web.json.scaffold; fixes it. The same applies to each row above. On the classpath nothing changes.

The POM’s optional spring-beans dependency is not named in the descriptor; no oumuamua-server-spi class references it. oumuamua-server-impl ships an explicit module descriptor that exports its four packages and re-exports org.smallmind.bayeux.oumuamua.server.api, org.smallmind.bayeux.oumuamua.server.spi, jakarta.servlet, jakarta.xml.bind, org.smallmind.nutsnbolts, and org.smallmind.scribe.pen. It requires org.smallmind.web.json.scaffold, which the startup configuration log uses, and opens org.smallmind.bayeux.oumuamua.server.impl to jakarta.xml.bind and tools.jackson.databind for the configuration view it writes. jakarta.websocket.client is requires static: an application that uses the WebSocket endpoint (websocket.WebSocketEndpoint) requires it, as in the spi table above. See Module Path And Classpath for the repository-wide rules.

All three artifacts are published under org.smallmind.

<dependency>
  <groupId>org.smallmind</groupId>
  <artifactId>oumuamua-server-api</artifactId>
  <version>${project.version}</version>
</dependency>

<dependency>
  <groupId>org.smallmind</groupId>
  <artifactId>oumuamua-server-spi</artifactId>
  <version>${project.version}</version>
</dependency>

<dependency>
  <groupId>org.smallmind</groupId>
  <artifactId>oumuamua-server-impl</artifactId>
  <version>${project.version}</version>
</dependency>
  • A Jakarta Servlet 6-compatible container (Jetty, Grizzly, Tomcat 10+). The repository’s test context uses smallmind-web-grizzly.

  • Java 25 or later. The default executor is Executors.newThreadPerTaskExecutor(…​) over a virtual-thread factory, wrapped with the resolved scope’s boundThreadFactory(…​) when the server has one (see Executor And Scope).

  • For WebSocket support: a JSR-356-compliant jakarta.websocket.server.ServerContainer (Tyrus, Jetty’s WebSocket bridge, or the Grizzly WebSocket runtime).

  • For clustering: an Apache Kafka broker reachable from every node and a smallmind-kafka-utility dependency on the classpath.

Note

The Kafka utility artifact is declared optional on the SPI module. Add it to your own application POM when you wire a KafkaBackbone; a single-node deployment can omit it.

The fastest way to stand up an Oumuamua server is to import the reference Spring XML shown below, register a servlet at /cometd/*, and let the servlet context listener publish the server into the container.

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
                           http://www.springframework.org/schema/beans/spring-beans.xsd">

  <bean id="oumuamuaServer" class="org.smallmind.bayeux.oumuamua.server.impl.OumuamuaServer">  (1)
    <constructor-arg>
      <bean class="org.smallmind.bayeux.oumuamua.server.impl.OumuamuaConfiguration">
        <property name="codec">                                                                 (2)
          <bean class="org.smallmind.bayeux.oumuamua.server.spi.json.orthodox.OrthodoxCodec">
            <constructor-arg>
              <bean class="org.smallmind.bayeux.oumuamua.server.spi.json.jackson.JaxbDeserializer"/>
            </constructor-arg>
          </bean>
        </property>
        <property name="protocols">                                                             (3)
          <list>
            <bean class="org.smallmind.bayeux.oumuamua.server.spi.websocket.jsr356.WebsocketProtocol">
              <constructor-arg name="longPollTimeoutMilliseconds" value="500"/>
              <constructor-arg name="websocketConfiguration">
                <bean class="org.smallmind.bayeux.oumuamua.server.spi.websocket.jsr356.WebsocketConfiguration">
                  <constructor-arg name="endpointClass"
                                   value="org.smallmind.bayeux.oumuamua.server.impl.websocket.WebSocketEndpoint"/>
                  <constructor-arg name="oumuamuaUrl" value="/cometd/*"/>
                </bean>
              </constructor-arg>
            </bean>
            <bean class="org.smallmind.bayeux.oumuamua.server.impl.longpolling.ServletProtocol">
              <constructor-arg name="longPollTimeoutMilliseconds" value="500"/>
            </bean>
          </list>
        </property>
        <property name="listeners">                                                             (4)
          <list>
            <bean class="org.smallmind.bayeux.oumuamua.server.spi.extension.AckExtension">
              <constructor-arg name="maxAckQueueSize" value="100"/>
            </bean>
          </list>
        </property>
        <property name="perApplicationScope" ref="org.smallmind.spark.perApplicationScope"/>    (7)
      </bean>
    </constructor-arg>
  </bean>

  <bean id="oumuamuaServerListenerInstaller"                                                    (5)
        class="org.smallmind.web.grizzly.installer.ListenerInstaller">
    <property name="contextPath" value="/smallmind"/>
    <property name="eventListener">
      <bean class="org.smallmind.bayeux.oumuamua.server.impl.OumuamuaServletContextListener">
        <property name="oumuamuaServer" ref="oumuamuaServer"/>
      </bean>
    </property>
  </bean>

  <bean id="oumuamuaServletInstaller"                                                           (6)
        class="org.smallmind.web.grizzly.installer.ServletInstaller">
    <property name="contextPath" value="/smallmind"/>
    <property name="displayName" value="cometd"/>
    <property name="servletClass" value="org.smallmind.bayeux.oumuamua.server.impl.OumuamuaServlet"/>
    <property name="urlPattern" value="/cometd/*"/>
    <property name="loadOnStartup" value="1"/>
    <property name="asyncSupported" value="true"/>
  </bean>
</beans>
  1. The server is built from a single configuration object. The constructor validates that a codec is present and that at least one protocol is defined.

  2. OrthodoxCodec is the bundled JSON codec. The JaxbDeserializer adapts Jackson into the codec’s deserialization pipeline.

  3. Both protocols are registered in the same list; the Bayeux handshake picks whichever the client asks for and the server supports. The longPollTimeoutMilliseconds is how long a held HTTP connection can sit idle before the server flushes an empty response.

  4. AckExtension is registered as a server-level listener; it attaches the acknowledgement-tracking logic to every session without further wiring. maxAckQueueSize is the cap on unacknowledged packets the extension will retain per session.

  5. The ServletContextListener publishes the server onto the servlet context under the well-known key org.smallmind.bayeux.oumuamua.server and stops it when the context is destroyed. The servlet and the WebSocket endpoint both pull the server back out by that attribute key; the servlet’s init calls start with its ServletConfig.

  6. The servlet installer mounts OumuamuaServlet at /cometd/*. The URL pattern must match whatever the WebsocketConfiguration.oumuamuaUrl declares, so that clients addressing /smallmind/cometd reach either transport depending on the connection type they initiated.

  7. The scope the server binds on the threads it starts and around its protocol-listener callbacks. The configuration only stores it; the OumuamuaServer constructor resolves it once, on the constructing thread (here the thread running the Spring refresh), through PerApplicationScope.resolve: the instance given is used when that thread has no scope bound or has this same instance bound, while a different bound scope fails construction with ScopeException (see Executor And Scope). Under the Tanuki wrapper with its scope on (spark.per.application.scope=true in wrapper.conf, or perApplicationScope on the plugin) this bean is registered by SpringContextWrapperListener during the refresh, which runs inside that same scope (see The Per-Application Scope); with the scope off no bean exists under that name and this ref fails the refresh with a BeanCreationException caused by NoSuchBeanDefinitionException, so leave the property unset. Any other host registers a singleton under that name before refreshing, or omits the property. An omitted property adopts the scope bound on the constructing thread — under the wrapper with its scope on, the same instance this ref names — and, with none bound either (the wrapper with its scope off among them), leaves the server without a scope: its threads then carry only the PerApplicationContext they inherit — the default executor’s threads from each submitting thread, the idle timers from the thread that called start. A KafkaBackbone (see Clustering With Kafka) resolves its own constructor argument the same way and must end up with this same instance. A backbone resolved to a different one — possible only on a constructing thread with no scope bound, when the two are handed different instances or only the backbone is handed one — delivers Kafka-received packets to the listeners inside the backbone’s scope, which holds none of the refresh’s registrations: without a context on its worker threads, DAO lookups return null, session lookups throw ORMInitializationException, and metrics are unplugged.

A browser client using CometD’s JavaScript library then connects with:

const cometd = new CometD();
cometd.configure({ url: "/smallmind/cometd" });
cometd.handshake();
cometd.subscribe("/chat/room/42", msg => console.log(msg.data));

Server-side, publishing a message to every subscriber of that channel takes a single line once the server bean is available:

Channel<V> channel = server.requireChannel("/chat/room/42");

ObjectValue<V> payload = server.getCodec().getFactory().createObjectValue();
payload.put("from", "server");
payload.put("text", "hello");

channel.publish(payload);
Type Role

Server<V>

The central contract. Owns channels, sessions, services, listeners, the codec, the security policy, the backbone, and the request/response/delivery pipeline. Exposes start(ServletConfig) / stop() for lifecycle, findChannel / requireChannel / removeChannel for channel management, onRequest / onResponse / deliver / forward for packet pipelining, and addListener / addService / addInitializer for extensibility.

Session<V>

A live client. Tracks SessionState (INITIALIZED → HANDSHOOK → CONNECTED → DISCONNECTED), long-poll vs. streaming delivery, the maximum long-poll queue size, and session-scoped packet listeners.

Channel<V>

A path-identified pub/sub destination. Tracks subscribers, channel listeners, and the persistent, reflecting, and streaming flags. Also defines Channel.WILD ("“) and `Channel.DEEP_WILD (”*"`) for pattern subscriptions.

Packet<V>

Immutable batch of one or more Message`s addressed to a `Route and classified by PacketType.

PacketType

Discriminant enum: REQUEST (client-to-server), RESPONSE (server reply), DELIVERY (fan-out).

Route

Parsed channel path. Exposes size(), getSegment(int), matchesPrefix(String…​), and the isMeta()/isService()/isWild()/ isDeepWild()/isDeliverable() predicates.

SessionState

Ordered lifecycle enum with comparison helpers (lt, lte, gte, ge) for readable state gates.

Protocol<V>

Named transport family. Owns Transport`s, exposes `isLongPolling() / getLongPollTimeoutMilliseconds(), and broadcasts receipt/publish/delivery events to `Protocol.ProtocolListener`s.

Transport<V>

Wire implementation beneath a protocol. Carries a name, an isLocal() flag (whether messages must not cross the backbone), and an init hook called once during server start.

BayeuxService<V>

Server-side RPC handler. Declares the literal /service/…​ routes it binds to and processes each request into a response packet; the helper createResponse builds a minimal reply with channel, id, and clientId pre-populated; the service sets successful and every other field itself.

SecurityPolicy<V>

Veto points on handshake, channel creation, subscribe, and publish. All four methods default to SecurityRejection.noReason(), a non-null deny sentinel, so inheriting without overriding yields a fully closed policy. Each operation must be explicitly opened by overriding the matching method to return null.

SecurityRejection

Any non-null SecurityRejection denies; a SecurityPolicy method must return null to allow. noReason() denies without explanation; reason(String) denies with an explanation propagated to the client.

ChannelInitializer<V>

A piece of configuration that runs the first time a given channel is created. Used to mark specific paths persistent, streaming, or reflecting. Registered via Server.addInitializer.

Attributed

Shared mixin giving Server, Session, Channel, and Transport a per-instance attribute map (getAttribute / setAttribute).

OumuamuaException / ChannelStateException / InvalidPathException

Typed failures: generic server-side failure, attempt to remove a persistent channel, and malformed channel path.

backbone.Backbone<V>

Cluster distribution contract: startUp(server), shutDown(), publish(Packet).

json.Codec<V> / json.Value<V> / json.Message<V>

JSON abstraction. Codec creates and parses messages; Value is the generic JSON-value supertype; Message is a ObjectValue with typed accessors for every field the Bayeux spec names (VERSION, ID, SESSION_ID, CHANNEL, SUCCESSFUL, ERROR, CONNECTION_TYPE, SUPPORTED_CONNECTION_TYPES, SUBSCRIPTION, EXT, ADVICE, DATA).

Type Role

AbstractProtocol<V>

Skeletal Protocol carrying the listener registry and the receipt/publish/delivery fan-out helpers so concrete protocols do not repeat the boilerplate.

AbstractAttributed

ConcurrentHashMap-backed implementation of Attributed shared by every core type that carries a dynamic attribute bag.

DefaultRoute / StringSegment / PathValidator

Canonical parsing of a slash-delimited Bayeux path into a Route plus its Segment list, with validation of permitted characters.

OpenSecurityPolicy<V>

The permit-everything SecurityPolicy used when the application does not supply its own.

Connection<V>

Transport-level abstraction binding a session to the protocol / transport handling its wire traffic.

Protocols / Transports

Enums of canonical wire names: Protocols.WEBSOCKET / Protocols.SERVLET, Transports.WEBSOCKET / Transports.LONG_POLLING. Used during handshake negotiation and for transport-locality decisions.

json.orthodox.OrthodoxCodec / json.orthodox.OrthodoxValue / json.orthodox.OrthodoxMessage / json.orthodox.*Value

The bundled JSON codec and its in-memory value hierarchy. Pluggable via a JsonDeserializer so that the serialization path can use either Jackson or a custom parser without touching the rest of the pipeline.

json.jackson.JaxbDeserializer

Jackson adapter satisfying JsonDeserializer for the orthodox codec.

json.CopyOnWriteArrayValue / json.MergingObjectValue / json.PacketUtility

Concurrency-aware JSON helpers used by the freezing pipeline and by extension authors who need to rebuild packets without mutating the originals.

extension.AbstractServerPacketListener<V>

Base class for any server-level PacketListener that wants the request / response / delivery fan-out already demultiplexed onto protected hooks.

extension.AckExtension<V>

Server-side implementation of the Bayeux ack extension. Tracks each session’s unacknowledged packets in a concurrent map, attaches ids, and re-queues any packet whose id is greater than the last acknowledged value.

extension.TimesyncExtension<V>

Server-side implementation of the Bayeux timesync extension: returns enough server-clock data on every response for a client to estimate one-way latency and clock skew.

backbone.kafka.KafkaBackbone<V>

Kafka implementation of the cluster backbone. Uses a shared topic named oumuamua-{topicName}; each node joins the topic with a unique, snowflake-generated consumer group so every node receives every record; records produced by the local node are filtered on receipt to avoid loopback.

backbone.DebonedPacket / backbone.RecordUtility

Wire-format serializer/deserializer for packets travelling over the backbone.

websocket.jsr356.WebsocketProtocol<V> / websocket.jsr356.WebSocketTransport<V> / websocket.jsr356.WebsocketConfiguration / websocket.jsr356.WebsocketConfigurator

JSR-356 WebSocket protocol, its transport, and the configuration objects that tell the JSR-356 server container which endpoint class to instantiate, at which URL pattern, and with which WebSocket extensions (e.g. permessage-deflate).

meta.Meta / meta.Advice / meta.Reconnect / meta.TransportUtility

Helper utilities for producing the advice block that the server ships on every meta response, advising clients on reconnect behaviour and transport fallback.

latency.InstrumentedLatencyListener

A reference ProtocolListener that records latency between receipt and delivery; useful as a template for metrics wiring.

Type Role

OumuamuaServer<V>

Concrete Server. Owns a ChannelTree, a ConcurrentHashMap of sessions, the protocol map, the service map, the listener list, and the initializer list. On start it brings the backbone up, initialises every protocol, and launches two maintenance threads.

OumuamuaConfiguration<V>

The single configuration record the server accepts. Lists every tunable described in Defaults Reference.

OumuamuaSession<V> / OumuamuaChannel<V> / OumuamuaConnection<V>

Concrete Session, Channel, and Connection implementations.

ChannelTree<V> / ChannelRoot<V> / ChannelBranch<V> / ChannelOperation

The channel tree itself, plus the visitor abstraction the idle sifter uses to walk it.

IdleChannelSifter<V> / IdleChannelOperation / IdleSessionInspector<V> / RemovedSessionOperation

The two maintenance threads. The channel sifter walks the channel tree every idleChannelCycleMinutes minutes and removes any non-persistent channel whose last-activity timestamp is older than channelTimeToLiveMinutes. The session inspector walks the session map every idleSessionCycleMinutes minutes and removes any session whose last-activity timestamp is older than sessionMaxIdleTimeoutSeconds.

OumuamuaServlet<V> / AsyncOumuamuaServlet<V>

The HTTP entry points for long-polling Bayeux traffic. Both look the server up off the servlet context, decode incoming messages through the server’s codec, hand the resulting packet to Server.onRequest, and stream the response back through the session’s queued packets.

OumuamuaServletContextListener<V>

ServletContextListener that publishes the server under Server.ATTRIBUTE at contextInitialized and invokes server.stop() at contextDestroyed; server.start(…​) is called by the servlet’s init.

longpolling.ServletProtocol<V> / longpolling.LongPollingTransport<V> / longpolling.LongPollingConnection<V>

HTTP long-polling side of the protocol/transport contract.

websocket.WebSocketEndpoint

JSR-356 endpoint. The JSR-356 container instantiates it per connection and calls its lifecycle methods; the endpoint looks the server up from the servlet context and bridges every WebSocket frame into Server.onRequest.

json.ClassNameXmlAdapter / json.ClassNameArrayXmlAdapter / json.DoubleStringArrayXmlAdapter

JAXB XML adapters used by the Doppelganger-generated configuration view when the configuration is serialized for diagnostic logging at startup. web-json-doppelganger itself is an optional, build-time-only dependency of oumuamua-server-impl; the generated view needs web-json-scaffold and jakarta.xml.bind-api, which the module declares directly.

This section zooms in on the small number of interfaces an application author is most likely to touch directly.

public interface Server<V extends Value<V>> extends Attributed {

  String ATTRIBUTE = "org.smallmind.bayeux.oumuamua.server";

  void start(ServletConfig servletConfig) throws ServletException;
  void stop();

  // channels
  Channel<V>   findChannel(String path)   throws InvalidPathException;
  Channel<V>   requireChannel(String path, ChannelInitializer... initializers)
                                          throws InvalidPathException;
  void         removeChannel(Channel<V> channel) throws ChannelStateException;

  // sessions
  Session<V>   getSession(String sessionId);

  // services
  void         addService(BayeuxService<V> service) throws InvalidPathException;
  void         removeService(Route route);
  BayeuxService<V> getService(Route route);

  // listeners and initializers
  void         addListener(Listener<V> listener);
  void         removeListener(Listener<V> listener);
  void         addInitializer(ChannelInitializer<V> initializer);
  void         removeInitializer(ChannelInitializer<V> initializer);

  // packet pipeline
  Packet<V>    onRequest(Session<V> sender, Packet<V> packet);
  Packet<V>    onResponse(Session<V> sender, Packet<V> packet);
  void         deliver(Session<V> sender, Packet<V> packet, boolean local);
  void         forward(Packet<V> packet);

  // configuration surface
  String       getBayeuxVersion();
  String       getMinimumBayeuxVersion();
  long         getSessionConnectionIntervalMilliseconds();
  boolean      allowsImplicitConnection();
  String[]     getProtocolNames();
  Protocol<V>  getProtocol(String name);
  Backbone<V>  getBackbone();
  SecurityPolicy<V> getSecurityPolicy();
  Codec<V>     getCodec();
  boolean      isReflecting(Route route);
  boolean      isStreaming(Route route);
}

Notable behaviours:

  • start runs once. It brings up the backbone, calls each protocol’s init exactly once with the ServletConfig forwarded so that transports that need to read servlet init-params can do so, and schedules the idle-maintenance executors. A second call throws ServletException before touching any of them; the servlet’s init is the one expected caller.

  • removeChannel throws ChannelStateException if the channel’s isPersistent() flag is true. Unsubscribe every session before removal if you want listeners to see the onUnsubscribed notifications.

  • deliver(sender, packet, local) is the main fan-out call. When local is true the packet originated on this node, and if a backbone is configured it is also handed to Backbone.publish for peer nodes. When the server receives a packet from the backbone, deliver is called with local=false (and a null sender) to prevent an infinite loop. The same sender and flag are passed through to every Server.PacketListener.onDelivery call.

  • forward(packet) is the server-initiated publish path behind Channel.publish(data) — useful for BayeuxService implementations and other server-side code that want to push data to a channel’s subscribers rather than send it directly back to a caller. It runs the same listener chain as deliver with a null sender and local=true, fans out to every matching channel (including / and /* wildcards), and always publishes to the backbone when one is configured.

Four subtypes, each tied to a distinct event:

Server.SessionListener

onConnected(Session) / onDisconnected(Session). Fires when a session has fully completed its handshake+connect and when a session has been disconnected (explicitly or by the idle inspector). With allowsImplicitConnection enabled, a handshaken session is also connected — and onConnected fires — by its first successful subscribe or publish; a rejected one leaves the session handshaken and fires nothing.

Server.ChannelListener

onCreated(Channel) / onRemoved(Channel). Fires when a channel is added to the tree (by requireChannel or by an implicit subscribe) and when it is removed (by removeChannel or the idle sifter). Registering a service creates no channel.

Server.SubscriptionListener

onSubscribed(Channel, Session) / onUnsubscribed(Channel, Session). Fires after every successful subscription change.

Server.PacketListener

Three transforming hooks. Each returns a (possibly replacement) Packet:

  • onRequest(sender, packet) — inbound client request.

  • onResponse(sender, packet) — outbound server response.

  • onDelivery(sender, packet, local) — fan-out packet, per subscriber. sender is the publishing session for packets published by a client on this node, and null both for packets received from the backbone and for packets published server-side through Channel.publish (which calls forward). local is true for every packet that originated on this node, client or server-side, and false for packets received from the backbone. Because the server-level chain runs on both the publishing node and every receiving node, a listener that must act exactly once cluster-wide should check local.

    Returning a different Packet replaces the original for every listener further down the chain. This is the mechanism AckExtension uses to splice in its tracking id; it is also how a custom auditing listener would redact sensitive fields before delivery.

public interface Session<V extends Value<V>> extends Attributed {

  String       getId();
  boolean      isLongPolling();
  void         setLongPolling(boolean longPolling);
  int          getMaxLongPollQueueSize();
  SessionState getState();

  void         completeHandshake();
  void         completeConnection();
  void         completeDisconnect();

  void         addListener(Listener<V> listener);
  void         removeListener(Listener<V> listener);

  Packet<V>    onResponse(Session<V> sender, Packet<V> packet);
  void         dispatch(Packet<V> packet);
  Packet<V>    poll(long timeout, TimeUnit unit) throws InterruptedException;
  void         deliver(Channel<V> fromChannel, Session<V> sender, Packet<V> packet);
}

The state transitions are monotonic: each complete* call moves the session forward once, and SessionState.lt / lte / gte / ge make it natural to gate operations on lifecycle position without hard-coding ordinals.

completeDisconnect() is idempotent — calling it on an already-disconnected session is a no-op and will not fire the disconnected callback a second time. It always signals any thread blocked in poll() so that long-poll connections drain promptly when a session ends.

poll() returns null when either the timeout expires or the session transitions to DISCONNECTED while the caller is waiting, whichever comes first.

dispatch bypasses the session’s packet listeners and hands the packet straight to the transport. deliver is the normal fan-out entry point — it runs the packet through the session’s listener chain, then either queues it for a long-poll transport or streams it.

public interface Channel<V extends Value<V>> extends Attributed {

  String WILD      = "*";
  String DEEP_WILD = "**";

  Route   getRoute();

  default boolean isWild();
  default boolean isDeepWild();
  default boolean isMeta();
  default boolean isService();
  default boolean isDeliverable();

  boolean isPersistent();       void setPersistent(boolean persistent);
  boolean isReflecting();       void setReflecting(boolean reflecting);
  boolean isStreaming();        void setStreaming(boolean streaming);

  boolean subscribe(Session<V> session);
  void    unsubscribe(Session<V> session);

  boolean isRemovable(long now);

  void    deliver(Session<V> sender, Packet<V> packet, Set<String> sessionIdSet);
  void    publish(ObjectValue<V> data);

  // listeners
  interface SessionListener<V>  extends Listener<V> { ... }
  interface PacketListener<V>   extends Listener<V> { ... }   // runs after freeze
}

The flag triad (persistent / reflecting / streaming) is the fastest per-channel customisation surface:

  • persistent: the channel is never removed by the idle sifter, even when empty. Use for channels the application manages directly.

  • reflecting: the publisher receives its own message as part of the fan-out. Default is off; chat-style echo is the typical use case.

  • streaming: the packet is dispatched immediately rather than queued. Matters only for long-poll transports; WebSocket transports always stream.

Because every listener level (Channel.PacketListener and Session.PacketListener) operates on frozen packets, a subscriber-specific personalisation at the session level is safe even though several sessions receive the same underlying packet.

A service channel (/service/foo) is not broadcast. Incoming messages are handed to the registered BayeuxService for that route and the returned packet is sent only to the caller. This is how you expose browser-to-server RPC over Bayeux.

public interface BayeuxService<V extends Value<V>> {

  Route[]    boundRoutes();
  Packet<V>  process(Protocol<V> protocol, Route route,
                     Server<V> server, Session<V> session, Message<V> request);

  default Message<V> createResponse(Route route, Server<V> server,
                                    Session<V> session, Message<V> request) { ... }
}

addService binds the service to every route in boundRoutes(). Each route must be a literal path beneath /service/: wildcard routes (/service/foo/, /service/*) and routes outside the service namespace are rejected with an InvalidPathException, and no route from that service is bound. The bare /service path cannot be bound either, because it is not a valid channel path at all (see Troubleshooting). Lookup is by exact route, so a request to /service/foo/bar reaches only a service bound to exactly /service/foo/bar; a request to a service path with no registered service is answered with successful: false and the error Unknown service.

createResponse returns a message with channel, id, and clientId already set. Everything else is the service’s responsibility, and the messages it returns are sent exactly as built: set successful: true on a reply, or successful: false with an error field on a failure — Bayeux requires successful on every response, and the server does not fill it in. A service may return null from process to send no response at all.

Service requests are not run through SecurityPolicy.canCreate or canPublish, and they are accepted from any session that has handshaken, whether or not it has completed /meta/connect. A service that needs either check performs it in process using the session and request it is handed.

A client /meta/subscribe naming a /service/…​ channel receives a successful response but is not recorded: no channel is created, SecurityPolicy.canSubscribe is not consulted, and nothing is ever delivered to the subscriber.

See Writing a Bayeux Service for a worked example.

public interface SecurityPolicy<V extends Value<V>> {
  default SecurityRejection canHandshake(Session<V> session, Message<V> message);
  default SecurityRejection canCreate   (Session<V> session, String path,  Message<V> message);
  default SecurityRejection canSubscribe(Session<V> session, Channel<V> channel, Message<V> message);
  default SecurityRejection canPublish  (Session<V> session, Channel<V> channel, Message<V> message);
}

All four methods default to SecurityRejection.noReason(), a non-null deny sentinel. The call sites treat any non-null rejection as a denial, so inheriting SecurityPolicy without overriding any method yields a fully closed policy. To permit an operation, an implementation must explicitly override the matching method and return null. Returning SecurityRejection.reason(String) denies the operation and propagates the reason back to the client in the error field of the matching meta response; returning SecurityRejection.noReason() denies without an explanation. The built-in OpenSecurityPolicy overrides all four methods to return null and may be wired in explicitly. When no policy is configured the server leaves the policy null, and the meta layer treats a null policy as allow-all.

None of the four hooks is consulted for /service/…​ traffic. A service request goes straight to the registered BayeuxService, which applies any authorization it needs inside process, and a subscription to a service channel is acknowledged without being recorded, so canSubscribe never sees it.

Nothing in the core API requires Spring. An application that prefers programmatic wiring constructs the configuration directly:

OumuamuaConfiguration<OrthodoxValue> configuration = new OumuamuaConfiguration<>();
configuration.setCodec(new OrthodoxCodec<>(new JaxbDeserializer()));
configuration.setProtocols(new Protocol[] {
  new WebsocketProtocol<>(500L, new WebsocketConfiguration(
    WebSocketEndpoint.class, "/cometd/*")),
  new ServletProtocol<>(500L)
});
configuration.setListeners(new Server.Listener[] {
  new AckExtension<>(100)
});

OumuamuaServer<OrthodoxValue> server = new OumuamuaServer<>(configuration);

servletContext.setAttribute(Server.ATTRIBUTE, server);
servletContext.addServlet("cometd", new OumuamuaServlet<>())
              .addMapping("/cometd/*");
servletContext.getServletRegistration("cometd").setAsyncSupported(true);

server.start(servletConfig);

The Spring XML in Quick Start is mechanical equivalent; use whichever idiom fits the rest of the application. The OumuamuaConfiguration class is annotated with @Doppelganger, so a JSON-pretty-printable view is logged at INFO on startup, which is useful for verifying that all the tunables resolved as expected.

A service channel handles request/response RPC from a browser client.

public class EchoService implements BayeuxService<OrthodoxValue> {

  private static final Route ECHO_ROUTE = new DefaultRoute("/service/echo");

  @Override
  public Route[] boundRoutes() {

    return new Route[] {ECHO_ROUTE};
  }

  @Override
  public Packet<OrthodoxValue> process(Protocol<OrthodoxValue> protocol, Route route,
                                       Server<OrthodoxValue> server,
                                       Session<OrthodoxValue> session,
                                       Message<OrthodoxValue> request) {

    Message<OrthodoxValue> response = createResponse(route, server, session, request);    // (1)
    response.put(Message.SUCCESSFUL, true);
    response.put(Message.DATA, request.get(Message.DATA));                                 // (2)

    return new Packet<>(PacketType.RESPONSE, session.getId(), route, response);            // (3)
  }
}
  1. createResponse pre-populates channel, id, and clientId using the originating request so the client can correlate the response. The service sets successful itself — true here; false plus an error field on a failure.

  2. The service copies the inbound data field back out — a textbook echo.

  3. The packet is tagged RESPONSE, which makes Server.onResponse run its listener chain rather than the delivery one, and the response goes to the caller only.

Register the service during application startup:

server.addService(new EchoService());

addService throws InvalidPathException if any bound route is a wildcard or lies outside /service/.

A listener that watches every message for auditing or instrumentation wires up as a Server.PacketListener:

public class AuditPacketListener<V extends Value<V>>
       implements Server.PacketListener<V> {

  @Override
  public Packet<V> onRequest(Session<V> sender, Packet<V> packet) {

    log.info("REQ [{}] -> {}", sender == null ? "anon" : sender.getId(),
             packet.getRoute().getPath());
    return packet;
  }

  @Override public Packet<V> onResponse(Session<V> sender, Packet<V> packet) { return packet; }
  @Override public Packet<V> onDelivery(Session<V> sender, Packet<V> packet, boolean local) { return packet; }
}

server.addListener(new AuditPacketListener<>());

Returning the same packet unchanged is the correct "observer" idiom. Returning a different packet transforms the stream for everyone downstream.

A Backbone is the cluster distribution layer; the bundled implementation speaks Kafka.

<property name="backbone">
  <bean class="org.smallmind.bayeux.oumuamua.server.spi.backbone.kafka.KafkaBackbone">
    <constructor-arg name="nodeName"                  value="app-node-1"/>  (1)
    <constructor-arg name="concurrencyLimit"          value="1"/>           (2)
    <constructor-arg name="startupGracePeriodSeconds" value="10"/>          (3)
    <constructor-arg name="shutdownGracePeriodSeconds"><null/></constructor-arg> (9)
    <constructor-arg name="groupProtocol"             value="CLASSIC"/>     (4)
    <constructor-arg name="dynamicConsumers"          value="true"/>        (5)
    <constructor-arg name="topicName"                 value="oumuamua"/>    (6)
    <constructor-arg name="perApplicationScope"       ref="org.smallmind.spark.perApplicationScope"/> (8)
    <constructor-arg name="servers">                                         (7)
      <bean class="org.smallmind.kafka.utility.KafkaServerFactoryBean">
        <property name="serverPattern" value="kafka-1:9094,kafka-2:9094"/>
      </bean>
    </constructor-arg>
  </bean>
</property>
  1. Identifies this node in log messages and is also stamped on every record this node produces, so each node can filter out its own records on receipt (loopback suppression). It must be unique per node. When dynamicConsumers is false it also prefixes each consumer worker’s Kafka group.instance.id (<nodeName>-<workerIndex>) and should then be stable across restarts of the same node.

  2. The number of background consumer workers polling Kafka on this node. One is enough unless inbound volume is very high.

  3. How long the constructor will wait for the broker to become reachable before giving up.

  4. The Kafka group protocol for the backbone’s consumers. Use CLASSIC for brokers that do not support KIP-848; use CONSUMER for Kafka 3.7 and later with the new consumer protocol enabled.

  5. Kafka group membership for the backbone’s consumers. true joins as dynamic members that leave the group as soon as they are closed; false joins as static members (group.instance.id = <nodeName>-<workerIndex>) whose assignments are held for the session timeout after they go away. Because every node already uses its own consumer group, static membership buys the backbone nothing; true is the sensible choice.

  6. The effective Kafka topic is oumuamua-{topicName}. Use a different topicName per Bayeux cluster if more than one independent cluster shares the broker.

  7. Bootstrap servers. KafkaServerFactoryBean resolves comma-separated host:port patterns into the producer/consumer bootstrap.servers property.

  8. Required argument with no scope-less overload; it precedes the servers varargs, so wire it by name as shown. The constructor resolves it once, on the constructing thread, through PerApplicationScope.resolve: the instance given is used when that thread has no scope bound or has this same instance bound; a different bound scope fails construction with ScopeException before any broker connection is attempted; <null/> adopts whatever scope is bound on the constructing thread. The consumer workers inherit no ScopedValue binding; when resolution leaves the backbone with a scope they run their whole poll loop — and so every packet they deliver to the local server — inside it. The backbone must end up with the same instance as the server configuration’s perApplicationScope: under the Tanuki wrapper with its scope on (spark.per.application.scope) that is the org.smallmind.spark.perApplicationScope singleton (see The Per-Application Scope), which <null/> also adopts because the refresh runs inside it; with the wrapper’s scope off no bean exists under that name — this ref fails the refresh with a BeanCreationException caused by NoSuchBeanDefinitionException — so pass <null/>, which then resolves to no scope; any other host registers a singleton under that name before the refresh, or passes <null/>. With no scope given and none bound, the workers carry only the PerApplicationContext they inherit from the thread running OumuamuaServer.start (the servlet’s init), which creates them in startUp. The producer’s executor only calls producer.send and is never wrapped.

  9. How long shutDown waits for queued send tasks to hand their records to the producer, and then how long again for the producer to flush buffered records to the broker before abandoning them. <null/> uses the value of startupGracePeriodSeconds. Like perApplicationScope, it precedes the servers varargs, so wire it by name as shown.

Omitting the scope through the same argument
<constructor-arg name="perApplicationScope"><null/></constructor-arg>

How it works, concretely:

  • The constructor checks broker connectivity and creates the one Kafka producer. startUp starts the consumer workers, each on a non-daemon platform thread with a Kafka consumer of its own (using a snowflake-generated consumer group per node, so every node independently sees every record); the JVM stays alive until shutDown.

  • publish(Packet) serializes the packet through DebonedPacket / RecordUtility and writes it as a record to the topic.

  • Each consumer worker polls the topic. For every record it receives, it compares the record’s node id against the local node’s id; matches are skipped to prevent loopback. Non-matches are deserialized and handed to Server.deliver(sender=null, packet, local=false) so that the fan-out happens without re-publishing to the backbone.

  • shutDown signals every consumer worker and wakes it via KafkaConsumer.wakeup(), then waits for each to exit, shuts down the send executor and waits up to shutdownGracePeriodSeconds for queued send tasks to hand their records to the producer (logging a WARN if they do not), then closes the producer with the same bound on flushing buffered records to the broker. Once shutDown has taken the stopping transition, the executor and producer are released even when a step throws or the thread is interrupted, and the backbone reaches STOPPED regardless; shutDown on a backbone that was never started releases them too. A shutDown that is still waiting for a concurrent transition releases nothing if it is interrupted there, or if the concurrent call was a startUp that completed (it then throws ComponentStateException and the backbone stays STARTED). The backbone is terminal afterwards: a later publish is dropped with a WARN rather than thrown, and a later startUp throws ComponentStateException. A send that fails — whether the producer throws or reports the failure through its callback — is logged at ERROR.

Important

The Kafka backbone fans every published packet to every node. Bayeux sessions and subscriptions are node-local state, so each node filters packets against its own subscription map. Plan topic capacity around total cluster message volume, not per-node volume.

Extensions in this codebase are simply Server.Listener implementations registered alongside your own listeners via the configuration’s listeners property. The two that ship out of the box are AckExtension and TimesyncExtension.

AckExtension implements the Bayeux ack protocol extension. On every delivery packet it assigns an ascending id, stores a copy of the packet in a per-session map keyed by that id, and on every subsequent client connect inspects the client’s echoed ack value to discard packets the client has confirmed. Anything that has not been acknowledged by the time a new long-poll window opens is replayed.

<bean class="org.smallmind.bayeux.oumuamua.server.spi.extension.AckExtension">
  <constructor-arg name="maxAckQueueSize" value="100"/>
</bean>

The maxAckQueueSize is a safety cap: if a client stops acknowledging but keeps reconnecting, the extension retains at most that many packets before it starts dropping the oldest entries.

Use AckExtension whenever a dropped delivery would be harmful — chat, alerts, order updates. Skip it when every delivery is already idempotent or when the client can just re-read the current state on reconnect.

TimesyncExtension implements the timesync extension. On every meta response it attaches server-clock information in the ext block; the client uses it to compute a clock-skew estimate and an approximate one-way latency. Useful for scenarios where the client’s display of "X seconds ago" needs to line up with the server’s clock — live status panels, auctions with deadline counters, etc.

  • requireChannel(path) creates the channel if it does not exist. Newly created channels have every registered ChannelInitializer applied before they are returned, so a subsequent requireChannel for the same path returns the already-initialised instance.

  • The idle channel sifter walks the tree every idleChannelCycleMinutes minutes and removes any channel whose isRemovable(now) returns true — non-persistent and past its channelTimeToLiveMinutes window since last activity. The actual removal is a two-phase operation: the sifter checks removability without the branch write lock, then re-verifies under the lock before terminating, closing the window where a subscriber could join between the outer check and the removal.

  • Persistent channels are never removed automatically; removeChannel on a persistent channel throws ChannelStateException. Clear the flag first if removal is intended.

  • A session goes INITIALIZED → HANDSHOOK → CONNECTED → DISCONNECTED monotonically. The three transitions correspond to /meta/handshake, /meta/connect, and /meta/disconnect (or the idle inspector).

  • The idle session inspector runs every idleSessionCycleMinutes minutes. For each candidate it atomically checks the idle timeout and transitions the session to DISCONNECTED in a single synchronized step, preventing a concurrent contact() call from keeping a session alive after eviction has already begun. A disconnected session fires its Server.SessionListener.onDisconnected and is removed from the session map.

  • Each session carries a long-poll queue sized by maxLongPollQueueSize. When the queue overflows, the oldest pending packet is dropped; a log line is emitted at the configured overflowLogLevel.

A single round trip walks the packet through several hands:

  1. Transport receives bytes, hands them to the codec.

  2. Codec produces one Message per JSON object; messages are grouped into a Packet by route, tagged REQUEST.

  3. Protocol.ProtocolListener.onReceipt fires.

  4. Server.onRequest runs the chain of Server.PacketListener.onRequest handlers. Any of them may return a replacement packet.

  5. The server decides whether this is a meta request, a service request, or a publish. Meta requests are handled internally; service requests are dispatched to the matching BayeuxService; publishes are fanned out via deliver.

  6. deliver runs Server.PacketListener.onDelivery (passing the publishing session as sender and local=true), freezes the messages, then walks every subscribed session and calls Session.deliver, which runs Session.PacketListener.onDelivery before queueing or streaming the packet.

  7. Responses walk the mirror path through Server.PacketListener.onResponse and Session.PacketListener.onResponse before being handed to the transport for sending.

Before a delivery packet reaches the channel-level listeners, its messages are frozen. Freezing means:

  • The packet instance is shared by every session in the fan-out.

  • A Channel.PacketListener or Session.PacketListener that wants to mutate a field is working on a copy (via PacketUtility) that is visible only to the current delivery stream.

The practical consequence: personalisation at the session level is cheap. Adding a per-client field to a delivery packet costs one copy per subscriber, not one cross-subscriber invalidation.

Route.isDeliverable()

Returns true for user channels (not wild, not deep wild, not meta, not service). Only deliverable routes may carry user publications.

Channel.isReflecting()

When true, the publisher’s own session receives its own message in the fan-out. Off by default; set via a ChannelInitializer or via OumuamuaConfiguration.setReflectingPaths for pattern-based configuration.

Channel.isStreaming()

When true, the packet is dispatched immediately rather than queued. Matters only for long-poll transports; WebSocket transports always dispatch immediately regardless of the flag. Configure broadly via OumuamuaConfiguration.setStreamingPaths.

OumuamuaConfiguration.setReflectingPaths(String[]) and setStreamingPaths(String[]) take an array of paths; each path is treated as a prefix tested against the route of a newly created channel via Route.matchesPrefix(String…​). A configured path matches every route whose leading segments line up with it — "" matches any single segment in that position and "*", when used as the final entry, consumes all remaining route segments. A candidate longer than the route never matches. The paths are parsed at configuration time and consulted by OumuamuaServer.isReflecting(Route) / isStreaming(Route) whenever a new channel is created, so newly created channels automatically inherit the broad rules without a per-channel ChannelInitializer.

  • OumuamuaServer is safe for concurrent calls on every public method. The session map, protocol map, and service map are ConcurrentHashMap; the listener and initializer lists are ConcurrentLinkedQueue; the channel tree is internally synchronised.

  • OumuamuaSession is safe for concurrent queue operations (long-poll producer/consumer) and for concurrent listener additions. completeDisconnect() is idempotent — calling it on an already-disconnected session is a no-op — and it signals any thread blocked in poll() so that the thread wakes immediately rather than waiting for its full timeout.

  • OumuamuaChannel protects its subscriber map with a per-channel monitor, but subscription and unsubscription callbacks (onSubscribed, onUnsubscribed) are invoked outside that monitor so that callbacks which re-enter the server do not deadlock. Channel termination via terminate() similarly returns the set of evicted sessions to the caller for post-lock callback dispatch.

  • AbstractAttributed uses a ConcurrentHashMap directly; no additional synchronization is applied, and getAttributeNames() returns an unmodifiable snapshot.

  • AckExtension uses a ConcurrentSkipListMap per session for the unacknowledged packet queue, so the per-session replay logic is safe against concurrent delivery and concurrent acknowledgement. On queue overflow the oldest (lowest-id) entries are evicted first.

  • `KafkaBackbone’s producer is thread-safe by design. Each consumer worker owns its consumer instance.

  • The default executor is a virtual-thread-per-task executor, meaning there is no practical ceiling on the number of in-flight delivery operations — the constraint is whatever the transport itself imposes.

The executor runs onMessages for long-polling requests and the process pass for WebSocket frames — the packet listeners, services, and SecurityPolicy calls those make. With no executorService configured the server builds a virtual-thread-per-task executor and, when it holds a scope, binds that scope on every task. A supplied executor is used as given, and its threads bind nothing:

  • With a scope and no PerApplicationContext on its threads, DAO and session lookups made from a supplied executor throw MissingPerApplicationContextException and Instrument metrics silently no-op.

  • With a context (the Tanuki wrapper, or any context-only host), a supplied executor whose threads were created from a thread carrying the context map resolves through it.

The server’s scope is fixed once, in the OumuamuaServer constructor, on the constructing thread. OumuamuaConfiguration.setPerApplicationScope only stores the value; the constructor passes it through PerApplicationScope.resolve, keeps the result, exposes it through getPerApplicationScope() (which AsyncOumuamuaServlet reads to build its executor), and applies it through runBound, which both servlets and the WebSocket endpoint use around their protocol callbacks. Resolution follows the ScopedValue binding of the constructing thread at that moment:

  • A scope given with no scope bound on the constructing thread, or with that same instance bound, is used as given.

  • A scope given while a different scope is bound fails the constructor with ScopeException.

  • No scope given (null, the default) adopts the scope bound on the constructing thread; with none bound either, the server has no scope, and its default executor, the idle-channel and idle-session timers, and the runBound callbacks run unbound, carrying only the PerApplicationContext they inherit.

Only the binding on the constructing thread counts; a scope bound later, or on another thread, is not adopted.

AsyncOumuamuaServlet never uses the configured executor: it builds its own virtual-thread-per-task executor in init, bound to the server’s scope when it holds one. OumuamuaServlet and the WebSocket endpoint submit to server.getExecutorService(), so a supplied executor affects only those two entry points.

To supply an executor that runs inside the scope, bind it from the scope instance. The static PerApplicationScope.wrapThreadFactory captures the binding of the thread that calls it and throws MissingPerApplicationScopeException when there is none, which is the case on a Spring refresh outside the wrapper or under the wrapper with its scope off; the instance methods boundThreadFactory and boundExecutor need no binding on the calling thread.

PerApplicationScope perApplicationScope = ...;                              (1)

configuration.setPerApplicationScope(perApplicationScope);
configuration.setExecutorService(Executors.newThreadPerTaskExecutor(
  perApplicationScope.boundThreadFactory(Thread.ofVirtual().factory())));   (2)
  1. The same instance the configuration, and any KafkaBackbone, receive.

  2. Every task the executor runs is bound to that scope.

Errors fall into three broad categories.

Path and configuration errors

InvalidPathException is thrown by Server.findChannel and Server.requireChannel when the path is syntactically invalid (empty segment, forbidden character). ChannelStateException is thrown by Server.removeChannel when the target channel is persistent. OumuamuaException wraps generic configuration errors at `OumuamuaServer’s constructor: missing codec, no protocols defined, missing configuration object.

Protocol errors

MetaProcessingException is raised inside the SPI when a meta message cannot be processed (unknown connection type, malformed subscribe, unsupported Bayeux version). The error is written back to the client in the matching meta response’s error field rather than propagated, so protocol errors never surface to listeners.

Security rejections

SecurityPolicy methods return SecurityRejection rather than throwing. The server inspects the rejection and, if populated, writes the rejection’s reason into the meta response’s error field and short-circuits the operation. No exception is raised.

Backbone errors

KafkaBackbone logs and swallows transient poll exceptions and reconstructs its consumer. A backbone whose constructor cannot reach the broker within startupGracePeriodSeconds throws a KafkaConnectionException, which aborts the Spring refresh that builds it (not OumuamuaServer.start).

Scope errors

ScopeException is thrown by the OumuamuaServer constructor when the configuration carries a PerApplicationScope and a different scope is bound on the constructing thread, and by the KafkaBackbone constructor under the same condition for its own argument. In a Spring assembly either aborts the refresh. See Executor And Scope.

Setting Default Source

channelTimeToLiveMinutes

30

OumuamuaConfiguration

sessionConnectIntervalSeconds

30

OumuamuaConfiguration

sessionMaxIdleTimeoutSeconds

120

OumuamuaConfiguration

idleChannelCycleMinutes

5

OumuamuaConfiguration

idleSessionCycleMinutes

1

OumuamuaConfiguration

maxLongPollQueueSize

1000

OumuamuaConfiguration

allowsImplicitConnection

false

OumuamuaConfiguration

messageLogLevel

TRACE

OumuamuaConfiguration

idleCleanupLogLevel

DEBUG

OumuamuaConfiguration

overflowLogLevel

DEBUG

OumuamuaConfiguration

executorService

A virtual-thread-per-task executor whose threads run inside the server’s resolved scope when it holds one. A supplied executor is used as given — its threads are the application’s to bind.

OumuamuaServer constructor

perApplicationScope

null — resolved once in the OumuamuaServer constructor, where null adopts the scope bound on the constructing thread; with none bound either, the server’s default executor, its idle-channel and idle-session timers, the servlets' protocol-listener callbacks, and the WebSocket endpoint’s delivery notification run with no scope, carrying only whatever PerApplicationContext is inherited: the default executor’s threads from each submitting thread, the timers from the thread that called start (the servlet’s init)

OumuamuaConfiguration

securityPolicy

none — a null policy is treated as allow-all

OumuamuaConfiguration

Long-poll timeout

Protocol-specific (test config uses 500ms)

Protocol.getLongPollTimeoutMilliseconds

Protocol names

"servlet", "websocket"

Protocols

Transport names

"long-polling", "websocket"

Transports

Kafka topic prefix

"oumuamua-"

KafkaBackbone

The extension surface is deliberate rather than accidental.

Custom SecurityPolicy

Override the hooks you need to open; all four defaults deny. Wire via OumuamuaConfiguration.setSecurityPolicy.

Custom BayeuxService

Implement BayeuxService<V>, register with server.addService(…​), bind one or more literal (non-wildcard) routes beneath /service/.

Custom Server.Listener

Four subtypes — session, channel, subscription, packet. Add via server.addListener(…​) or the OumuamuaConfiguration.listeners property.

Custom ChannelInitializer

Run logic the first time a particular channel is created. Wire via server.addInitializer(…​); initializers apply in registration order.

Custom Protocol / Transport

Subclass AbstractProtocol; implement Transport directly. The protocol declares its name (matched during handshake negotiation), whether it uses long polling, and the pool of transports it owns.

Custom Backbone

Implement Backbone.startUp / Backbone.shutDown / Backbone.publish. The implementation is responsible for invoking Server.deliver(null, packet, false) for every remote packet that should reach local sessions.

Custom Codec / Value

Implement Codec<V> and a Value<V> hierarchy. The only constraint is that Message<V> must be ObjectValue<V>-assignable and expose the standard Bayeux field constants.

Handshake never returns

The servlet is not mounted at the URL the client is calling, or the WebSocket URL pattern in WebsocketConfiguration.oumuamuaUrl does not match the servlet mapping. Confirm that the URL the client uses is exactly the servlet’s URL.

Client subscribes but never receives

Check whether the channel is marked persistent. A non-persistent channel whose TTL has expired is removed, and so are its subscriptions. Also check the server log for a SecurityPolicy rejection reason in the meta-subscribe response.

ChannelStateException on removal

The channel is persistent. Clear the flag first (channel.setPersistent(false)) or unsubscribe and let the idle sifter retire it.

InvalidPathException

The path is malformed. Valid paths start with a slash, contain non-empty segments of allowed characters, and end with a segment (not a trailing slash). Use or * only as the final segment. The bare paths /meta and /service are reserved namespace roots and are rejected wherever a path is parsed: a client that publishes, subscribes, or unsubscribes on either receives an error response naming the path, and findChannel / requireChannel throw rather than create them. addService also throws it when a bound route is a wildcard or is not beneath /service/; the service is then not registered on any route.

Long-poll queue overflow

A session is being produced to faster than it can consume. The oldest packet is dropped and a log line is emitted at overflowLogLevel (default DEBUG). Either raise maxLongPollQueueSize, enable streaming on the flooded channels, or reconsider the rate at which the application publishes.

Cluster peers never receive a published message

Verify the KafkaBackbone topicName is identical on every node, and that the bootstrap servers resolve to the same Kafka cluster. Check the per-node consumer group ids in the broker to confirm each node is in fact a distinct group (every node should appear).

Packet listener is never invoked

Confirm that the listener was registered before the server started (for @Doppelganger-serialised configurations, make sure it is in the listeners property). Listeners added after start work for subsequent traffic only.

Single-cluster Kafka topology

KafkaBackbone assumes every node participates in the same Kafka cluster and the same topic. Cross-cluster federation is out of scope.

No JSR-356 auto-registration

The WebSocket endpoint is wired through WebsocketConfiguration and installed by the application’s servlet container configuration; the server does not scan for @ServerEndpoint classes.

No annotation-driven service scan

BayeuxService`s must be registered explicitly through `server.addService(…​) or the configuration’s services property. There is no classpath scanner.

Single-transport-per-session

A session is bound to one transport for its lifetime. If a client needs to upgrade from long polling to WebSocket, it disconnects and handshakes again.

Pattern subscription is subscribe-time

/foo/ and /foo/* are literal channel paths a client can subscribe to — the server treats them as wildcard subscriptions on fan-out. There is no retroactive subscription: a client that subscribes to /foo/* after /foo/bar has already been published will not see the past publication.

oumuamua-server-api defines the Bayeux contract. oumuamua-server-spi supplies the base classes, the orthodox JSON codec, the ack and timesync extensions, the Kafka backbone, and the JSR-356 WebSocket protocol. oumuamua-server-impl is the actual running server — the channel tree, the servlet and WebSocket entry points, the long-polling protocol, and the idle-cleanup threads.

Wire OumuamuaConfiguration with a codec, one or more protocols, and any extensions you need; hand it to OumuamuaServer; install OumuamuaServletContextListener and OumuamuaServlet in your servlet context at /cometd/*; expose WebSocketEndpoint at the same URL for WebSocket clients. Subscribe, publish, and let the server carry the messages.