The author | ZhangYe Ming

Dr. Almond CTO. Middle-aged programmer who focuses on various technologies and team management.

This article is based on the 10.22 GDG talk “Why Should Startups Embrace containers?” To sort out.

1. What is a container?

In the last article (A Startup’s Path to containerization (I) – Before Containerization), we briefly explained the architecture development before containerization of almond. Today we are going to talk about containers.

Docker came out of nowhere in 2013, and gradually came into people’s eyes in 2015. Containers don’t have to be Docker, of course, and containers now have standards. But when it comes to containers, Docker comes to mind. So when we’re talking about containers, we’re basically talking about dockers.

What exactly is a container? As the name suggests, containers are for holding things. In this case, it’s an application. There are four characteristics of containers:

  1. A container is self-contained; it packages the application and all its dependencies and can be run directly. Dependency management for applications has been a big problem in the past, and even though things like RPM, Maven, Ansible, and others can solve some of these problems, there was no standard mechanism for all applications until containers came along.

  2. Containers are portable and can behave the same way almost anywhere. This ensures that the application has exactly the same running environment in development, test, production, and so on.

  3. Containers are isolated from each other. Multiple containers running on the same host do not affect each other. That is, an application running in one container cannot access resources (processes, networks, files, users, etc.) in other containers unless configured as shared resources.

  4. Containers are lightweight, manifest in the second startup of containers, and consume very few resources.

A virtual machine can do a lot of things that containers can do, so what’s the difference? The image below is a screenshot from Docker’s official website, which perfectly illustrates the difference between the two.

But the fundamental difference, in fact, is the last point: light weight. Many people might think it’s a simple distinction, but it’s not. This is what makes containers a standardized way to distribute applications.

Last century 5, 60 time container just appeared, look also just simple difference, also do not have what technical content. However, container provides a standardized logistics method, the global sea, land and air transportation, dock loading and unloading around the container formed a whole efficient logistics system. Ultimately changing world trade and contributing to globalization.

So the container, the standardized way of application distribution, ultimately affects the entire application architecture of the upper layer. Ultimately, a complete application architecture will be built around the container, bringing about revolutionary changes. We can already see that Kubernetes has become a standard, and Google recently released Istio, a Service Mesh tool, to further abstract away the microservices infrastructure.

2. What is container choreography?

It’s not enough to have a container that can hold your applications, and if you still manage so many containers manually, you won’t be able to take advantage of the container. So we need a container choreography system. Container choreography provides the following functions:

  1. Application scheduling: Application deployment, seamless upgrade, elastic expansion, and self-healing.

  2. Resource management: memory, CPU, storage space, and network.

  3. Service management: namespace, load balancing, and health check.

  4. And many other features, such as logging, monitoring, authentication, authorization, and so on.

The three major systems in container choreography are Docker Swarm, Kubernetes, and Marathon/Mesos.

Docker Swarm is the official Docker program, the advantage is simple, the disadvantage is too simple.

Then there’s Google’s Kubernetes, or K8s. Kubernetes has dominated the containerization space in recent years, and Docker officially announced its support for Kubernetes. It has the advantage of being supported by big manufacturers, and it is strategically positioned by Google. You can think of it as Android in those days; It also has a hot neighborhood.

Technically, Kubernetes is an integrated solution, well designed, and a model of distributed system design. Google has a lot of experience in this area. The downside is that it is a bit complex, and until this year it had a number of issues, including performance issues, compatibility with larger versions, and complex deployment, which have now been largely resolved.

Mesos predates Docker and manages resources for distributed systems. Mesos is flexible enough to support a wide range of systems, including Spark. Marathon implements choreography based on Mesos.

We looked at containerization in the middle of last year, and our final solution was Marathon/Mesos. One reason is that Jenkins containerization has used Marathon/Mesos before, and has some experience. On the other hand, the solution is easy to integrate with the current architecture. Kubernetes is too complex, and migrating would change the architecture too much.

2. Containerization of almonds

Our containerized architecture looks like this:

All applications run on Mesos Slaves as containers, and the Mesos Master centrally manages the Mesos Slave server. Marathon uses the Mesos scheduling container for publishing, upgrading, and expanding. Calico is the network solution of Docker, which implements one IP container and the interconnection between containers. In the upper right corner, we basically keep our original microservices architecture, except that the Consul Agent for service discovery and registration has been replaced with Registrator.

Our CI/CD has also been adjusted accordingly.

Jenkins himself is now container-based and will compile and package in the Mesos Slave container. The app will be packaged as a Docker image and uploaded to Harbor, our mirror warehouse. When deploying, Jenkins calls Marathon’s interface for deployment, and Marathon applies for resources from Mesos. Mesos downloads the appropriate application image from Harbor at deployment time and runs it according to the configuration.

With this system, it’s easy to create and extend applications. When creating the application, first create the image through Dockerfile and Jenkins, then on Marathon interface, only need to prepare a Json configuration (also through Form configuration), specify resources, instance number, image, network, health check, environment variables, etc. You can launch a new app very quickly.

Sometimes we need to prepare a split-kill campaign or push millions of users, add an app instance, and just adjust a number in Marathon.

In addition to container Choreography and CI/CD, there are two very basic things, one is a unified logging platform based on ELK and the other is a monitoring and alarm platform based on Open-Falcon, StatsD, Graphite, Grafana. The general structure is as follows, specific implementation after the opportunity to write a special article, here is not detailed.

Ultimately, our entire platform looks like this:

4. Container summary

So far, that’s the architecture of almond’s current base platform. With this system, we have improved resource utilization, and when we first moved to a containerized environment, we only used about 60% of our original cloud servers. And we have greatly strengthened our ability to automate operations and improve service monitoring.

But there are problems with the system.

  • Adding server nodes still requires some manual work.

  • The configuration of containers, environments, and so on is scattered everywhere and lacks effective management.

  • Poor support for stateful applications.

  • The system has some redundancy, such as Zookeeper, Etcd, and Consul.

  • Automatic capacity expansion is not supported.

  • Parts of the infrastructure are not containerized.

We will continue to evolve in the future, perhaps migrating to container services in the public cloud or building our own Kubernetes cluster when the time is right.

The full text after


You may also be interested in the following articles:

  • Lego micro Service Transformation (I)

  • Lego micro service Transformation (II)

We are looking for a Java engineer. Please send your resume to [email protected].

Almond technology station

Long press the left QR code to pay attention to us, here is a group of passionate young people looking forward to meeting with you.