General introduction

Easy-serverless is an application deployment platform, used to meet the urgent needs of many colleagues to deploy applications, deployment of small applications no longer need: Apply for server, build environment, apply for network permission, apply for domain name and a series of operations, open the Easy-Serverless platform, you can through a few steps to deploy the application to the company’s internal Kubernetes cluster. At present, it supports the deployment of front-end static websites, Node service, Nginx service, Mongodb and Redis database, MinIO object storage, etc.

The platform relies on the company’s internal Kubernetes cluster. Each type of containerized application is created from a custom image, using apis provided by Kubernetes’ Javascript client library to perform container operations. Persistent storage uses the Longhorn distributed storage solution to manage storage volumes.

User analysis

Proportion of service types

According to the statistics when the article was published, the total number of running services is 114


It can be seen that most platform users choose to deploy static front-end websites such as personal blogs and platform official websites on the easy-Serverless platform.

Overall Architecture

The token is the token of the service account created in the cluster, and the access permission is assigned. You can obtain the corresponding token by using kubectl get secret.

The user-defined namespace in the figure indicates the namespace created by the user. The services created by the user run in their own namespace.

Background knowledge

To understand the working principle of easy-serverless, we need to first understand the knowledge of Docker, Kubernetes, Longhorn, those with relevant background can skip this chapter.

Docker

Docker has similar functions to virtual machines, but is more lightweight. It does not need to simulate hardware or run an independent operating system. It only provides completely isolated Containers for each application, and different containers are not affected by each other. The three important concepts in Docker are:

  • dockerfile
  • image
  • container

Dockerfile defines the image, which can be specified as a Linux base image, or as an open source image on Docker Hub, and can then be customized on the base image. Once the image is defined, different containers can be created from the image, and the environment of the container created is consistent (such as operating system, Node version, etc.). Compared with deploying the same application on multiple different servers, creating multiple container applications reduces the trouble of deployment and avoids inconsistent application performance caused by inconsistent server environments.

Kubernetes

Kubernetes is an open source Docker container choreography system. A Kubernetes cluster usually contains two or more servers, including a master node and multiple nodes. The master node is responsible for the management and control of the whole cluster. It can well support distributed systems, including fault discovery, service rolling upgrade, online capacity expansion and other capabilities. All resource objects in Kubernetes can be defined using yamL files.

Some of the most important concepts in Kubernetes are:

  • Namespace (namespace)
  • pod
  • service
  • kubectl

  • namespaceImplements resource isolation and logically distinguishes different resources. ineasy-serverlessThis specifies the namespace to which resources such as pods, storage volumes, and so on belong. In addition, the Namespace can be configured with configMap to set resource quotas, such as the number of services and storage space.
  • podThere will be one or more containers, which is where our service will run. You can use the YAML file to define the corresponding Deployment based on a particular image, or you can define the resource type POD directly.
  • serviceIs independent of the POD lifecycle and can expose services inside the POD to the outside of the cluster.KubernetesThe inside of thekube-proxyIs a load balancer that forwards service requests to a POD instance at the back end. The service also has its own unique Cluster IP.
  • kubectlFor the clientcliA tool that allows users to perform operations, including viewing clusters, on the CLIpodRunning status, resource object creation, modification, running, etc., we can provide through the websiteInteractive tutorialTo learn about andKubernetesThe basic operation of.

Longhorn

Longhorn is a Kubernetes distributed persistent volume management system. Longhorn and Ceph are two open source solutions in the community.

Kubernetes has its own persistent volume, longhorn provides a relatively complete management system, you can run the Longhorn UI to manage the storage volume. When applying for resources using Longhorn, you also need to define persistent storage volume declarations in YAML. For a two-node cluster, each volume corresponds to an engine and two copies on different nodes, ensuring that each node can access the volume and that the orchestration of the volume copies depends on Kubernetes.

Function point analysis

After understanding the basic knowledge involved, the following will be the implementation of the specific functions of easy-Serverless.

Image customization

Easy – ServerLesss supports the creation of static websites, Node services, Mongo, minio, and other services based on custom images uploaded to the corporate cloud. This section uses creating a simple Node service image as an example to describe how to create a customized image.

(1) Install Docker locally and define your own Dockerfile

FROM node:16-alpine3.11
     
MAINTAINER  username [email protected]

SHELL ["/bin/sh"."-c"]

RUN echo 'start build' && \
  sed -i 's/dl-cdn.alpinelinux.org/mirrors.yourmirror.com.cn/g' /etc/apk/repositories && \
  apk add git
Copy the code

Dockerfile Custom image based on DockerHub image node: 16-Alpine3.11, alpine different versions of the node image, the default has installed node, NPM, YARN, etc. Considering the network restriction of accessing the official mirror source directly, the source address of the mirror is changed to the internal source address of the mirror, and some customized operations are performed.

(2) Build your image locally based on Dockerfile

Docker build-f./dockerfile -t my-node:16-alpine3.11.Copy the code

(3) After the successful construction, you can run a container in the local whale application Docker Desktop to test related functions

(4) The company’s cloud platform needs to package the image and upload it. We use the save command to save the compressed package to the local, and then upload it to the company’s cloud platform, so we can use our own image

docker save -o ./node-16-alpine.tar.gz  my-node
Copy the code

Creation of services and applications

We use the @kubernetes-client-node client library to use the capabilities provided by Kubernetes, which provides a series of apis to operate and manage our services. In addition to JavaScript client, official also provides Go, Java, Python, dotnet and other client libraries for easy use.

As mentioned earlier, all resource objects in Kubernetes can be defined using YAML files. Resource object definitions include Pod, Deployment, Service, Ingress, Replication Controller, etc. A simple YAML definition is as follows:

apiVersion: XXX
kind: XXX                     // type
metadata:                     // Basic information
  name: XXX
  labels: 
    {{key: value}}
spec:                         // Define the options in detail
  container: 
  volumes:     
Copy the code

Taking a node service created by the platform as an example, we analyze which resources need to be declared to deploy a service. In the easy-Serverless implementation, there are altogether three resource types defined: ConfigMap (configuration set), Deployment (Deployment, including container). And Service.

  • ConfigMap: A set of configuration information, a combination of environment variables, which can be found through the API or the command line kubectl describe ConfigMap app_name.

  • Deployment: Deployment also contains some basic information, creating a container in the spec details and performing operations such as pulling repository code, installing dependencies, running services, and defining internal ports.

  • Service: checks that if an external port is defined, the external port can be accessed. If no external port is defined, the internal port can access the unique Cluster IP address.

With these three parts of YAML written, we can use the API to create services from the YAML file.

Quota configuration

To prevent abuse of server resources, some quotas are configured in easy-serverless.

Each user is allocated a certain amount of resources when registering, such as the number of services allowed to be created and the available storage capacity. When creating a resource, the system compares the used quota with the allocated quota to determine whether the user has enough quota to add resources.

Manage storage volumes

For applications like mongodb and Redis, we need persistent storage.

Longhorn is characterized by its support for distribution. Longhorn-ui provides a visual interface to operate storage volumes. There are corresponding restart, backup, snapshot and other functions.

To access Longhorn-UI, find the external port number of Longhorn’s service and use the IP address of the master node :port

Domain name and ingress configuration

Domain name we have applied for XX.dev.XXX.net. When you edit the domain name, you can set traffic forwarding on the interface. In fact, an ingress type is created.

Ingress is an internal function supported by Kubernetes. Similar to Nginx, ingress can be configured to forward resources, but only to internal cluster services. Rules in ingress are added based on user-configured forwarding rules.

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: {{domainName}}
  labels:
    create-by: ezsvs
spec:
  rules:
  - host: {{domainName}}.dev.XXX.net
    http:
      paths:
      {{#each forwards}}
      - path: {{path}}
        # pathType: Prefix
        backend:
          serviceName: {{serviceName}}
          servicePort: {{servicePort}}
      {{/each}}
Copy the code

The resources

Book: The Definitive Guide to Kubernetes

Dockerhub Kubernetes official documentation longhorn official documentation