Run multiple DBMS and C# containers behind the HAProxy load balancer with Docker Compose. Test scalability with a different number of instances

In the last article, you created a microservices architecture and manually implemented application layer database sharding.

Now you will extend the application and run multiple container instances of microservices and databases. You will use Docker Compose and a HAProxy load balancer.

Then you run the JMeter load test to see how the application scales with different numbers of instances. Finally, you will publish and receive messages from RabbitMQ.

1. Run multiple database and microservice instances

Dockerize micro service

Use the code and environment from the previous article as a base.

Rename the file” Dockerfile “in Visual Studio Project-Explorer to “Dockerfile” (first character lowercase). Then right-click the Dockerfile and choose “Create Docker Image”. This will also push the image to docker.

Run the application in Docker

Create the file _docker-comemage.yml_.

Medium.com/media/46c01…

The Docker file is configured with three database containers and four instances of the _Post_ service. Currently, only two databases are used by the _Post_ service instances. You can later remove comments to use a third database. A HAProxy load balancer exposes the _Post_ service container on port 5001.

Each container has a CPU limit of 0.5 to aid in actual load testing on the local machine. On my 12-core laptop, I still have unused resources, so adding more services and database instances can be beneficial.

Start the application

C:\dev>docker-compose up -d
Copy the code

Start the database

On http://localhost:5001/swagger/index.html, open your browser

Start the database with at least 100 users and 10 categories.

You can create more users and categories, but this will take some time due to CPU limitations.

2. Load test the extended application with JMeter

Create a JMeter test plan

Install and open JMeter.

Create a test plan and a thread group.

Thirty-two threads is a good number to start with. In each loop of a thread, it adds one post and reads 10 posts.

Add an HTTP request to create a post.

  • Server name: localhost
  • Port: 5001
  • HTTP-Request:POST
  • The path. /api/Posts
  • Principal data.
{
Copy the code

It creates a post for a random user (ID 1-100) and category (1-10).

Add a Content-Type application/json HTTP header to the request.

Read 10 posts in a random category.

  • Server name: localhost
  • Port: 5001
  • HTTP-Request:GET
  • The path. /api/Posts
  • Send parameters with the request.
NAME | VALUE | CONTENT-TYPE
Copy the code

Run the test

Take a look at the _ summary report _ while the tests are running.

There should be no mistakes.

Wait for a while until the _ mean _ (response time) and _ throughput _ values become stable.

Modifying test Parameters

Stop testing in JMeter.

You can change the threads in the test plan. Increase them to, say, 64 or 128 threads. Or reduce the number of threads to 16, or even one.

Close the application before editing _docker-comemage.yml_.

C:\dev>docker-compose down
Copy the code

You can change the number of service instances by using the “scale” property. Change the environment property of the number of databases (add/remove comments).

After your changes, launch the application.

C:\dev>docker-compose up -d
Copy the code

It takes some time until the database server is running. And remember to initialize the database.

3. Sample test results

** On my computer, the ratio of two job services to one database can get good results. ** I can scale it up to six services and three databases until I reach the limits of hardware. The average time remained below 500ms. Increasing the number of threads above 64 produces an error.

These results depend on my environment and CPU limitations. It will be different on your machine, too.

Throughput per second is proportional to the number of instances.

With limited CPU, 305 requests per second, that’s about 25 million requests per day. This will allow 1 million users to write 10 posts and read posts per day. Of course, real life applications are more complicated, but I hope this example illustrates the basic idea.

4. Communication between microservices and replication of user changes

The Post service receives user changes from the _User_ microservice via Rabbitmq messages.

You will also simulate this with JMeter.

Create the RabbitMQ container

Issue the following command (one line in the console window) to start the RabbitMQ container with an administrative UI.

C:\dev>docker run -d  -p 15672:15672 -p 5672:5672 -e RABBITMQ_DEFAULT_USER=test -e RABBITMQ_DEFAULT_PASS=test --hostname my-rabbit --name some-rabbit rabbitmq:3-management
Copy the code

This command configures _”test_” as a user and password instead of the default Guest_ credential _. Guest is limited to localhost, but within docker, containers are on different hosts.

Example Modify the Post microservice

Install the _rabbitmq. Client_**NuGet package in Visual Studio.

Add _IntegrationEventListenerService_ class.

Medium.com/media/6d77b…

The background service uses the _”test_” account to access RabbitMQ and uses host_.docker. Internal and _ localhost as hosts. This allows you to connect from inside the container to the Visual Studio debugger.

A single active consumer guarantees that only one _Post_ service instance receives messages.

If the active instance crashes, the next instance takes over. You can try this later by stopping the currently received instance in docker.

This code creates exchanges and pipes if they don’t already exist. It uses manual validation.

* * to * * modify the _Startup cs_ IntegrationEventListenerService operation.

Medium.com/media/7f3c3…

Run the changed microservice in Docker

Close the application in Docker.

C:\dev>docker-compose down
Copy the code

Build the _Post_ service, dockerize it and publish it to the Docker.

Launch the application after your changes.

C:\dev>docker-compose up -d
Copy the code

This will take some time until the database server is running. And remember to initialize the database.

Modify JMeter tests

Add a single thread group to the JMeter test plan.

Add a constant timer.

A constant timer limits tests to one message per second.

One message per second is pretty frequent. Even if 1,000,000 users sign up in half a year, that’s only four new users per minute.

Add an HTTP request to publish messages to RabbitMQ.

  • Server name: localhost
  • Port: 15672
  • HTTP-Request:POST
  • The path. /api/exchanges/%2F/userloadtest/publish
  • Physical data.
{
Copy the code

The user entity has a version field to handle out-of-order messages. To keep the test simple, it updates only one user and adds the version field. The performance impact should remain constant.

Add an HTTP header manager for _ content type _ and authorization to RabbitMQ.

Content-Type | application/json
Copy the code

“DGVzdDp0ZXN0” is the base64 encoded user and password “test”.

Run the test

Throughput should be similar to the previous tests.

You can also identify active RabbitMQ consumers in docker by looking at console output. You can stop the container and another instance will take over.

5. Final thoughts and prospects

You created a microservices architecture and implemented application layer database sharding. You then extend the application with multiple container instances and load test it. You also handled user change events from RabbitMQ.

This is just a sample application. You will have to adapt the code to use it in production.

What’s next? Retrieving data like the top 10 categories with the most posts requires querying multiple database instances. This can cause delays and kill performance. Redis can be a solution for querying aggregated data. I’ll show it in the next article.

You can also run the application in Kubernetes. See my other articles.

  • How to deploy your ASP.NET Core application into Kubernetes
  • Use Angular for the UI

If you have any questions, ideas or suggestions, please contact me.

How to extend ASP.NET Core microservices and hash databases. Load testing with JMeter was originally published on ITNEXT’s Medium, where people continue the conversation by highlighting and responding to the story.