Perf QBit versus Spring Boot

First question people usually ask me. How does QBit compare to X?

Where X is the favorite framework of the person or something they once read an article about.

Often times that X is Spring Boot.

So how does QBit compare, keep in mind, we have not done any serious perf tuning of QBit. We are more focused on features, and you could use QBit on a Spring project. I have used QBit running inside of Spring Boot. QBit is a lib. It can run anywhere. QBit can run with Guice. You can run QBit inside of Vert.x.

Out of the box QBit REST compares to out of the box Spring REST as follows:

QBit code

package hello;


import io.advantageous.qbit.annotation.RequestMapping;

import java.util.Collections;
import java.util.List;

import static io.advantageous.qbit.server.EndpointServerBuilder.endpointServerBuilder;

/**
 * Example of a QBit Service
 * <p>
 * created by rhightower on 2/2/15.
 */
@RequestMapping("/myservice")
public class MyServiceQBit {

    @RequestMapping("/ping")
    public List<String> ping() {
        return Collections.singletonList("Hello World!");
    }

    public static void main(String... args) throws Exception {
        endpointServerBuilder()
                .setPort(6060)
                .build()
                .initServices(new MyServiceQBit()).startServer();
    }

}

Spring code

package hello;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.context.annotation.Scope;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.ResponseBody;
import org.springframework.web.bind.annotation.RestController;

import java.util.Collections;
import java.util.List;

@RestController
@EnableAutoConfiguration
@Scope
public class MyServiceSpring {

    @RequestMapping(value = "/services/myservice/ping",
            produces = "application/json")
    @ResponseBody
    List<String> ping() {
        return Collections.singletonList("Hello World!");
    }

    public static void main(String[] args) throws Exception {

        SpringApplication.run(MyServiceSpring.class, args);
    }

}

The code looks similar. Keep in mind that QBit only does JSON over HTTP or WebSocket. Spring supports a lot more types of content. Also keep in mind that QBit can happily run inside of Spring. QBit is just a library.

But, people ask.

Now how do they compare when you are talking about performance.

Spring Boot

$  wrk -c 2000 -d 10s http://localhost:8080/services/myservice/ping
Running 10s test @ http://localhost:8080/services/myservice/ping
  2 threads and 2000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    10.03ms    2.12ms  31.37ms   67.26%
    Req/Sec    13.25k     1.61k   16.33k    65.50%
  264329 requests in 10.05s, 46.43MB read
  Socket errors: connect 0, read 150, write 0, timeout 0
Requests/sec:  26312.11
Transfer/sec:      4.62MB

Here we see that Spring Boot handles just over 26K TPS. This is just running it with wrk on OSX. You can expect higher speeds with a properly tuned Linux TCP/IP stack.

QBit RAW

$ wrk -c 2000 -d 10s http://localhost:6060/services/myservice/ping
Running 10s test @ http://localhost:6060/services/myservice/ping
  2 threads and 2000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    21.61ms   18.09ms 553.68ms   99.37%
    Req/Sec    48.78k     3.20k   52.86k    93.50%
  971148 requests in 10.02s, 80.58MB read
Requests/sec:  96910.31
Transfer/sec:      8.04MB

As you can see, QBit runs well over 3x faster. In fact it is approaching 4x faster. (Wait to see what happens when we start optimizing this.)

Now you are thinking but this is not a fair test. You are right.

QBit has to funnel everything through the same thread. Spring does not.

QBit ensures that only one thread at a time runs the ping operation so you can put stateful things inside of the QBit services. Thus QBit has to do a lot more, and it is still 3x faster.

The underlying network speed is coming form Vert.x which QBit uses as its network transport toolkit.

QBit also lets you easily run the same service as a WebSocket, and when you do that, you will get much higher throughput.

• QBit WebSocket speed 580K TPS (running on OSX macbook pro)
• QBit HTTP speed 90K TPS (running on OSX macbook pro)
• Spring Boot REST 26K TPS (running on OSX macbook pro)

Now this does not mean much. But I have built very real things with QBit and Vert.x. Things which handles millions and millions of users using a fraction of servers as other similar services.

QBit is a lot more than this silly perf test. QBit has a health system, a stats system, integrates with Consul, integrates with StatsD, and much more.

• QBit implements a easy-to-use reactive API where you can easily manage async calls to N number of other systems.
• QBit allows sharded in-memory services for CPU intensive apps.
• QBit allows pooled/round-robin in-memory services for IO intensive apps.
• QBits stats system can be clustered so you can do rate limiting across a pool of servers.
• You can tweak QBit's queuing system to handle 200M+ TPS internal queuing.
• QBit has a high-speed, type safe event bus which can connect to other systems or just run really fast in the same process.

It is no mistake that QBit annotations look a lot like Spring MVC annotations. I spent a lot of years using Spring and Spring MVC. I like it.

It also no mistake that you do not need Spring to run QBit. This allows QBit to be used in projects that are not using Spring (they do exist).