Actix versus Fiber: performance comparison in database operations

Introduction

A couple of weeks ago I came across a well-written article by Mayank Choubey, where he did a performance comparison between a Web API written in Go using the Fiber framework and the Actix framework written in Rust.

In it, it transpired that Fibre was considerably faster than Actix, which seemed strange to me. A couple of months I tried comparing Go and Rust in a Web API setting (using Actix and Gin, instead of Fibre). So, my curiousity got the better of me and I decided to test it myself.

To test this, and to make sure my own machine would not get in the way of measurements, I deployed the two frameworks on a small Azure Kubernetes Cluster.

Test setup

To test this, I decided to use the following repositories:

• The Rust Web API: https://github.com/snoekiede/eventsrustapi
• The Go Web API: https://github.com/snoekiede/eventswebfibre

Both repositories contain a Dockerfile and in the kubernetes directory the necessary yaml file to set up a small Kubernetes cluster, which for my tests ran on a Standard_DS2_v2 virtual machine. Note that for simplicity, the postgres database is also part of the Kubernetes cluster, which is usually not recommended.

To fill the database, I used Postman to run a request a 100 times, so there were 100 items in the database.

For the final loadtesting I used Locust using the following simple ‘locustfile.py’:

from locust import HttpUser,task
from random import randrange

class EventUser(HttpUser):
    
    @task
    def get_event_list(self):
        self.client.get("/api/events")

    @task
    def get_random_event(self):
        id = randrange(1,50)
        self.client.get(f"/api/events/{id}")    

In this you can see I test two things:

1. The complete lists of events
2. And a random event from the list

My choice for Locust has to do with the fact that it has a nice web interface out-of-the-box, which makes it easy to run tests and visualize the results.

Note that for the Fiber API, I replace ‘/api/events/’ with ‘/events’ and ‘/api/events/{id}’ with ‘/event/{id}’

Test methodology and caveats

I tested with following number of users:

• 20 users
• 100 users
• 250 users
• 1000 users

Caveat 1: This is a very naive approach, I realize that

Caveat 2: Running load tests from my local machine has some drawbacks:

1. I am running on quite an old Windows machine with 12G of memory, this may affect measuremens
2. Also because I am running the test locally on a remote cluster, some network latency must also be taken into account
3. The code both in Go and Rust could probably be more efficient, given my relative inexperience with these languages and the tested frameworks
4. Because I am using a Postgresdb database, the used ORM’s may affect performance too. I use Diesel for Rust and Gorm for Go.

Well, time to start testing

The Fiber framework

Developing the API

Developing in Go is usually a pretty painless process, and this was no exception. In a previous experiment I had used Gin so the basic principle were pretty clear, only the names of some methods were different. Also the fact that the handler method had to return an error was new to me.

The API Performance

First we look at the performance with 20 users:

Well, more than 400 RPS doesn’t seem to bad, and the performance seems to be stable.

Time to put some pressure on the API:

Still not a bad performance, although the response time seems to have increased slightly

Well, how about 250 users?

Not a big fall in the average performace, but from the graph we can tell that the performance on average might be good, it is starting to show some instability

Also, the first failures start to show up:

These might be due to the the limitations of the Kubernetes cluster or due to the fact that I am testing from my local machine.

Well, time for the ultimate test: 1000 users

The performance remains rather stable, just above 300 RPS on average, however, the instability has grown, and we have more failures:

The cause of these errors is not clear, perhaps I ought to research this.

The Actix framework

Developing the API

Developing in Rust always takes somewhat more time. The API you see in the repository took me about a month to develop, because

1. I had to get used to Rust’s module system
2. Diesel is powerful, yet not always straightforward

To be fair Actix is not that hard to learn, and yet quite powerful.

The API Performance

Like with the Fiber API, let’s not put too much pressure on it, and start with 20 users:

Not bad, almost 600 RPS on the same configuration as the Fiber API. Also the performance seems to be very stable. This may be due to the fact that Rust does not have a garbage collector.

Since this went so well, a bit more pressure should not hurt: 100 users:

To my suprise the performance got somewhat better, over 600 RPS!. Also the performance again is very stable.

Time to turn up the notch a bit more: 250 users.

Again about the same number of RPS, somewhat over 600 RPS, a very stable performance to. Unfortunately some failures start to appear:

The cause for these failures is not clear, it maybe due to limited resources of the Kubernetes cluster.

Also for Actix the ultimate test: 1000 users

A very stable performance, Actix really can take this amount of beating, still over 600 RPS on average. However, like with Fiber, some failures start to appear:

As mentioned before, this may be due to the limited resources of the Kubernetes cluster.

Conclusion

From the figures and graphs above and considering the aforementioned caveats, one may conclude that Actix performs quite a lot faster and remains far more stable under pressure than Fiber.

However, keep in mind this was a very naive test done out of sheer curiousity and for a specific usecase. This test is by no way definitive, just informative.

Should I use Go or Rust for my next backend project?

This question again has no definitive answer. The answer depends on a number of factors:

• Your (team’s) familiarity with either language
• The importance of performance for your project. Some projects are very performance-critical
• The Developer Experience (DX). Or in other words: how easy is it to pick up a language and get results?

So, the choice of language depends on your team and your usecase. Also keep in mind that Rust and Go are not the only options for a backend service.