Nate, Vice President of IT at Active Network, introduces the tools they use to build the Kubernetes platform. He shares his experience and advice on building a stable, reliable, and convenient K8S platform.


Kubernetes has become the industry’s accepted container choreography fact standard, and it is almost ubiquitous. IT is hard to meet an IT practitioner who has never known about Kubernetes. Software development, testing, and infrastructure fall within its “business scope.”

Before you can create a production-ready Kubernetes platform, you need to be ready to migrate your application from a traditional environment, which is by no means an easy task. You need an ETCD database, kuBE-Controller, Kube-Scheduler, certificates, core-DNS, etc. It also invests resources and time in research to determine the right combination of components and reliable, adaptive testing methods that can change quickly when components are swapped or enhanced.

In addition to creating the environment and managing processes, it is important to plan the company’s strategy and tactics. Few companies are starting to adopt containers simply to meet infrastructure needs. It is more likely that your container environment design and technology stack will need to support old and new forms of computing while avoiding reusing assets, resources, and expenses. (Examples of technology stack and design functions are shown below)





Management and automation


Rancher is an open source enterprise Kubernetes management platform, for enterprise users to provide Kubernetes-as-A-Service (Kubernetes is a Service), its simple and intuitive interface style and operation experience, The problem of poor usability and steep learning curve of Kubernetes native UI has been solved to a great extent. By using Rancher, you can take the pain out of manually creating Kubernetes clusters and automatically set them up with one click. It also provides a set of capabilities required by a K8s cluster, including configuration, access control, global DNS, disaster recovery and recovery, monitoring, logging, and cluster upgrades. You can configure CentOS VMs for K8s nodes using Ansible.

Logging and Monitoring


Rancher has a built-in FluentD deployment that can be used to build EFK stacks. Each cluster can be configured to push FluentD logs to Elasticsearch instances.


Kibana is an open source visualization platform that makes it easy to view and search Elasticsearch logs.


Elasticsearch making repo:

https://github.com/helm/charts/tree/master/stable/elasticsearch


Prometheus is a good solution for collecting monitoring indicators. Prometheus Server can be used to store time series data, alert-Manager can be used to manage alarms, and Node-Exporter can export indicators from nodes. Kube-state-metrics can generate metrics for all K8S objects.


Prometheus making repo:

https://github.com/helm/charts/tree/master/stable/prometheus


However, Prometheus lacked a user interface and therefore required Grafana, a data visualization tool. It connects to Prometheus Server and provides charts and dashboards for monitoring.


Grafana making repo:

https://github.com/helm/charts/tree/master/stable/grafana


Persistent storage


Everything in Kubernetes is dynamic and stateless, which goes against the principles of traditional storage schemes. Therefore, choosing a viable persistent storage solution is one of the challenges you will face. There are many popular solutions on the market, such as Ceph, Rook, StarageIO, and Portworx.


Among them, Portworx has data mobility, high availability, platform independence, and dynamic encryption configuration for persistent volumes. On the worker node, we recommend anotherDisk (VMDK) to create a storage pool using Portworx. Portworx comes with an intelligent scheduler called Stork that can save on licensing costs by installing Portworx on just a few working nodes.


You can deploy Portworx using the helm Chart:

https://github.com/portworx/helm/tree/master/charts/portworx

The container security


Container security is evolving all the time, and due to the dynamic nature of PODS, it is critical that all processes and communications occurring within the container be visible and controllable. Neuvector provides continuous runtime protection for hosts and pods by scanning Kubernetes clusters, nodes, pods, and container images to protect containers from security vulnerabilities. There is also the added advantage of being able to provide Docker and Kubernetes benchmarks for clusters. It can also become a network firewall by learning the good behavior of POD/Service and dynamically create security policies based on this. When a Service is in “protected mode,” it prevents any unauthorized process or network traffic from running for the POD or service.


You can deploy Neuvector using the helm Chart:

https://github.com/neuvector/neuvector-helm


Load balancing


Once applications are deployed in a K8S cluster, there are several options to expose them outside the cluster. Another consideration is if you are migrating your application from a traditional infrastructure to a container and want to preserve the rollback state or to preserve services in a traditional environment that have now been migrated to a K8S cluster.


AVI Networks provides a software-defined load balancer with a control plane and a service plane. This load balancer provides load balancing, traffic management, elastic scaling, and end-to-end automated K8S services. AVI deploys the service engine as PODS on the K8S cloud, which handles north-south traffic (that is, traffic between clients and servers) as well as load balancing for K8S services.


Each time ingress is created in the K8S cloud, AVI is configured with a DNS server and IPAM pool to automatically create virtual services. It assigns an IP from IPAM, creates a DNS entry, and configures a pod pool on the back end. AVI can also add various HTTP policies and network security policy functions through ingress annotations.


CI/CD tools


Because everything and its dependencies are packaged into containers, Kubernetes can make continuous deployment a reality, and it can schedule workloads from specific work nodes. A rolling update strategy enables continuous deployment with no downtime.


Jenkins is an excellent continuous integration and mirror building tool that provides integration with Gitlab, Nexus, JFrog ArtiFactory, SonarQube, Neuvector, Fority, Helm, and Rancher. To form a complete CI/CD assembly line.


Helm packages the entire application stack into chart, including pods, services, secert, ingress, persistent storage, and so on required by the application. Helm also enables consistent deployment across different environments. (As shown below)



All in all, there are now many ways to help build the Kubernetes platform, and deploying applications in A K8S cluster is easier than ever. I hope this gives you a better idea of what areas to focus on and what tools or platforms can make K8S clustering a reality in the enterprise.