This article describes how to install Traefik V2 in K3S 1.20. If you want to upgrade V1 to V2, you can refer to the official Migration Guide: From V1 to V2

K3s cluster uses Traefik as Ingress Controller by default, K3S 1.20 and earlier install Traefik V1 by default, and K3S 1.21 and later install Traefik V2 by default. The V2 version has significant performance and functionality improvements.

Our cluster is K3S 1.20, and Traefik V1 is installed by default. In consideration of performance and function richness, we plan to upgrade to Traefik V2. Because our cluster is a new one and does not involve compatible modifications of the existing ingress, we will uninstall and reinstall it. For the upgrade, see the official Migration Guide: From V1 to V2 to modify the incompatible areas one by one.

The following are the installation highlights and problem logs.

The installation

We used Helm to install, and you can check out the latest supported versions from Rancher’s app Store.

If the default configuration meets the requirements, you can install it on the previous page. To customize the helm Chart configuration, you can Download the Download link in the lower right corner of the page. The installation name is the same as the page name.

There is also an official way to customize the default installed components, using HelmChartConfig to add a custom configuration in the configuration list directory. Personally, I feel that it is not as straightforward as the direct customization of helm Chart. You can try the official scheme.

We use the downloaded Helm Chart for a custom installation:

Write your own values to customize part of the configuration.

The following is the values file k3s-values.yaml that I customized during installation.

# Create dashboard IngressRoute by default # enable dashboard IngressRoute by default # enable dashboard IngressRoute by default # enable dashboard IngressRoute by default # ingressRoute: # dashboard: # enabled: false deployment: enabled: true # Can be either Deployment or DaemonSet kind: Deployment # Number of pods of the deployment (only applies when kind == Deployment) replicas: 2 ports: traefik: port: 9000 # You SHOULD NOT expose the traefik port on production deployments. # If you want to access it from outside of your  cluster, # use `kubectl port-forward` or create a secure ingress # kubectl port-forward $(kubectl get pods --selector "app.kubernetes.io/name=traefik" --output=name -n cattle-system) 9000:9000 -n cattle-system expose: false exposedPort: 9000 web: port: 8000 expose: true exposedPort: 80 websecure: port: 8443 expose: true exposedPort: 443 tls: enabled: Metrics: port: 9100 Expose: true exposedPort: 9100 affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: app.kubernetes.io/name operator: In values: - traefik topologyKey: kubernetes.io/hostnameCopy the code

2. Install using helm

Yaml # helm upgrade --install traefik./ -n cattle-system -f k3s-values.yaml cattle-system -f k3s-values.yamlCopy the code

This is good, but before giving the above configuration, I encountered several problems, for reference, to avoid pit mining.

The problem record

1. How to configure Dashboard?

Two methods are available. The first method is port-forward. The steps are as follows:

  • 1, through theingressRouteCreate a route,helm chartIf you want to customize it, you can set it to false and create it yourselfingressoringressRoute.
  • 2, through theport-forwardBinding port
kubectl port-forward $(kubectl get pods --selector "app.kubernetes.io/name=traefik" --output=name -n cattle-system) 9000:9000 -n cattle-system
Copy the code

Access to the local localhost: 9000 / dashboard/can, behind dashboard/must have, otherwise you’ll be at 404.

The second is to expose the port directly, and the steps are as follows:

1. Modify values as follows:

ports:
  traefik:
    port: 9000
    expose: true ## <<< is changed to true and defaults to false
    exposedPort: 9000
Copy the code

Run the following command to update the configuration:

helm upgrade --install traefik ./ -n cattle-system -f k3s-values.yaml
Copy the code

3. The traefik service exposes port 9000. We can use Rancher to create a normal ingress.

IP host domain name binding cluster nodes, can be accessed through http://traefik-prod.example.com/dashboard/#/.

The first option is officially recommended for safety reasons.

2. How to support TLS?

There is a bit of a glitch in the Charts as follows:

  websecure:
    port: 8443
    expose: true
    exposedPort: 443
    tls:
      enabled: true  #<< is disabled by default, you need to enable it, otherwise the ingress does not support TLS certificate configuration
Copy the code

Ports. The websecure. TLS. Enabled by default is false, this leads to configure a custom certificate is not effective in ingress. This parameter must be turned on when the HTTPS ingress is required.

3. What is the problem with traefik deployment mode in K3S?

Ingress can be deployed in three ways in K8S:

  • 1. IngressController was deployed as DaemonSet, exposing hostPort;
  • Deploy as Deployment, open a NodePort service for IngressController.
  • 3, Deploy as Deployment, expose a LoadBalancer service to IngressController (need to use the cloud provider capability provided by the cloud vendor to assign IP);

In K3S, a Load Balancer named Klipper Load Balancer is used. It will run a POD on all nodes by default with daemonset (it will start an error when the node configuration is spotty, which needs to be configured by itself) as a proxy service. Therefore, in K3S, the Deployment effect of TraefiK is the same as that of DaemonSet Deployment.

Comparison of TRAeFIK deployed in the second mode of K8S is as follows:

4. Traefik deployment copy problem?

In K8S, the node where traefik is deployed can receive traffic. However, as Klipper Load Balancer is used in K3S, all nodes can receive traffic by default. However, SVCLB, a daemonset, relies on Traefik. When Traefik is restarted, SVCLB is unavailable. You are advised to configure at least two Traefik pods and enable the anti-affinity function to prevent the PODS from being deployed on the same node to improve availability.

deployment:
  enabled: true
  # Can be either Deployment or DaemonSet
  kind: Deployment
  # Number of pods of the deployment (only applies when kind == Deployment)
  replicas: 2  ## <<<< at least two copies with POD anti-affinity enabled.

affinity: 
  podAntiAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchExpressions:
            - key: app.kubernetes.io/name
              operator: In
              values:
                - traefik
        topologyKey: kubernetes.io/hostname        
Copy the code

5. Traefik Monitoring configuration?

The monitoring interface for metrics in the Charts is not enabled, so we can modify the ports.metrics. Expose parameter to enable as follows:

  metrics:
    port: 9100
    expose: true  # <<< is changed to true and defaults to false
    exposedPort: 9100
Copy the code

Then add a serviceMonitor and you’ll see the target in Prometheus.

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: traefik
  namespace: cattle-monitoring-system
  labels:
    app.kubernetes.io/name: traefik
spec:
  jobLabel: app.kubernetes.io/name
  selector:
    matchLabels:
      app.kubernetes.io/name: traefik  # service discovery label
  namespaceSelector:
    matchNames:
    - cattle-system  # traefik install namespace
  endpoints:
  - port: metrics
    interval: 30s
    path: /metrics
Copy the code

conclusion

Most of the components in Rancher can be installed by customizing Helm Chart, as long as the resource names are consistent. You can install it using the Chart package in Rancher’s App Store, create the CRDS your app needs, and install it using Helm Chart.

I’m DeanWu, a person trying to be a real SRE.

Follow the public account “Mr. Wu code nong”, you can get the latest articles in the first time.