Wechat official account: Operation and maintenance development story, author: Jock

In Kubernetes, the Service IP and Pod IP are primarily used for internal cluster access and are not visible outside the cluster.

If you want to access it from outside the cluster, you usually do so in the following three ways.

(1) Use NodePort

The diagram below:

image.png

Ignoring the implementation details, NodePort is used to map a port to all nodes. The default port range is 30,000-32767. The user then accesses the interior of the cluster through any host + mapped port.

The advantages of this method are that it is relatively simple to implement. The disadvantages are that if there are too many ports mapped, it is difficult to manage. Each port can only provide one service, and there are many iptables rules generated from this method.

(2) Use the LoadBalancer mode

The diagram below:

image.png

As can be seen from the figure, the Load Balancer service exposure is very direct. The service is directly exposed to the Internet, and the traffic destined for the specified port is forwarded to the corresponding service. It has no filtering conditions, no rules, etc.

The biggest disadvantage of this approach is that each LoadBalancer requires a fixed IP and, more often than not, cloud vendor support.

(3) Through Ingress

The diagram below:

image.png

The Ingress Controller is deployed as a Pod in a Kubernetes cluster. In essence, we cannot access it directly from outside, but we still need to expose it in several ways:

  • Exposure via NodePort is preceded by a load balancer

  • Exposure via LoadBalancer is the default for cloud vendors

  • Use hostPort directly in Pod, followed by a load balancer

Ingress is smarter and friendlier, but slightly more complex to configure. It exposes multiple applications with one IP address, supports different URIs with the same domain name, and supports certificates.

Ingress currently exposes services within a cluster in a recognized best way within the industry. However, due to its importance, there are many Ingres controllers in the world. Common ones are:

  • Kubernetes Ingress

  • Nginx Ingress

  • Kong Ingress

  • Traefik Ingress

  • HAProxy Ingress

  • Istio Ingress

  • APISIX Ingress

In addition to those listed above, there are also many Ingress controllers. Facing so many Ingress controllers, how should we choose? What are the criteria for reference?

In general, it can be judged from the following dimensions:

  • Supported protocols: Whether to support protocols other than HTTP(S)

  • Routing rules: Which forwarding rules exist and whether regex is supported

  • Deployment policy: Supports ab deployment, Canary deployment, and blue-green deployment

  • Upstream probe: What mechanism is there to determine if the application is healthy or not, is there a solution for active and passive checking, retries, fuses, custom health checks, etc.

  • Load balancing algorithm: Supported load balancing algorithms, such as Hash, sticky session, RR, and WRR

  • Authentication mode: What authorization schemes are supported? Basic, digest, OAuth, external authentication, etc

  • DDoS defense capability: Supports basic rate limiting and whitelist

  • Full link tracing: Indicates whether full link monitoring can be connected

  • JWT validation: Whether there is built-in JSON Web token validation to authenticate and validate users of the end application

  • Graphical interface: Whether a graphical interface is required

  • Custom scalability: whether it is easy to expand

The following is a brief introduction to the above Ingress Controller.

Kubernetes Ingress

github.com/kubernetes/ingress-nginx

The official recommended Ingress controller for Kubernetes Ingress, which is based on the Nginx Web server and complements a set of Lua plug-ins for additional functionality.

Because of the popularity of Nginx, the Ingress controller is the easiest controller to use after migrating applications to K8S, and the learning cost is relatively low. If you don’t have high requirements for controller power, it is recommended to use it.

However, Reload is slow when there are too many configuration files, and while there are many plugins available, the plugin scalability is very weak.

Nginx Ingress

github.com/nginxinc/kubernetes-ingress

Nginx Ingress is the official version of Nginx development. It is based on the Nginx Plus commercial version. The Nginx controller has high stability, continuous backward compatibility, no third party modules, and high speed (compared to the official controller) due to the elimination of Lua code.

Compared with the official controller, it supports TCP/UDP traffic forwarding, the paid version has a wide range of additional functions, the main disadvantage is the lack of authentication, traffic scheduling and other functions.

Kong Ingress

github.com/Kong/kubernetes-ingress-controller

Kong Ingress builds on NGINX and adds a Lua module that extends its functionality.

Kong, which previously focused on API gateways, has now become a mature Ingress controller. Compared to official controllers, kong has improved routing matching rules, upstream probes, authentication, and supports a large number of module plugins, which are easily configurable.

It provides some API, service definition, can be abstracted into Kubernetes CRD, through Kubernetes Ingress configuration to complete synchronization state to Kong cluster.

Traefik Ingress

github.com/containous/traefik

Traefik Ingress is a fully functional Ingress. It is officially called: Traefik is an Edge Router that makes publishing your services a fun and easy experience.

It has many useful features: continuous configuration updates (no reboot), support for multiple load balancing algorithms, Web UI, metric export, support for various protocols, REST apis, Canary versions, and more. Out-of-the-box “Let’s Encrypt” support is another nice feature. With support for TCP/SSL, Canary deployment and traffic mirroring/shading in version 2.0, the community is very active.

Istio Ingress

istio.io/docs/tasks/traffic-management/ingress

A joint project of IBM, Google, and Lyft (Envoy’s original author), Istio is a comprehensive service grid solution. It can not only manage all incoming external traffic (as the Ingress controller), but also control all traffic within the cluster. Behind the scenes, Istio uses Envoy as a secondary proxy for each service. Essentially, it’s a large processor that can do almost anything. The central idea is maximum control, scalability, security and transparency.

With Istio Ingress, you can fine-tune traffic routing, access authorization between services, balancing, monitoring, canary publishing, and more.

However, Ingress Gateways is now recommended by the community.

HAProxy Ingress

github.com/jcmoraisjr/haproxy-ingress

As an ace load balancer, HAProxy has the greatest advantage in load balancing among many controllers.

It provides “soft” configuration updates (no traffic loss), DNs-based service discovery, and dynamic configuration through the API. HAProxy also supports full customization of configuration file templates (by replacing ConfigMap) and the use of Spring Boot functions in them.

APISIX Ingress

github.com/api7/ingress-controller

ApiSix Ingress is a new Ingress Controller that mainly targets Kong Ingress.

It has very powerful routing ability, flexible plug-in expansion ability, performance is also very good. At the same time, its disadvantages are also obvious. Although APISIX has a lot of features after open source, there is no implementation case, no relevant documentation to guide people how to use these features.

A comparison is attached below:

Photo from Tencent cloud community share

There is a more complete contrast scheme: docs.google.com/spreadsheet…

Write in the last

Each controller has its own advantages and disadvantages. In addition to referring to the above standards, the selection should also be considered:

  • Technical staff stack, maintenance ability

  • The true appeal of the company’s business

What fits the company is the best, not for the sake of use, but because of need.

Reference links:

  • kubedex.com/ingress/

  • Juejin. Cn/post / 684490…

  • zhuanlan.zhihu.com/p/87735361

Public account: Operation and maintenance development story

Making:Github.com/orgs/sunsha…

Love life, love operation