On April 29, 2021, the KubeSphere open Source community is excited to announce the release of KubeSphere 3.1.0. In order to help enterprises maximize the efficiency of resource utilization, KubeSphere has created a cloud native distributed operating system with Kubernetes as the kernel, which provides a plugable open architecture and seamless connection with third-party applications, greatly reducing the threshold for enterprise users.

KubeSphere V3.1.0 features “edge-to-edge container hybrid cloud” with new support for “edge computing” scenarios. At the same time, on the basis of V3.0.0, metering and charging are added to make the operating cost of infrastructure clearer, and further optimize the use experience in “multi-cloud, multi-cluster, multi-team, multi-tenant” and other application scenarios. Enhanced features such as “multi-cluster management, multi-tenant management, observability, DevOps, app store, micro-service governance”, and further improved interaction design to enhance user experience.

Cloud native industry alliance to write “cloud native development white paper mentioned, everything connected age accelerated the cloud – synergy evolution of demand, the traditional centralized storage cloud computing center, calculation model has been unable to meet the terminal equipment for aging, capacity and the demand of the work force, to the edge of the sink and the unified delivery, with the center operations, controls, Has become an important development trend of cloud computing.

In response to this trend, KubeSphere has worked closely with the KubeEdge community to expand Kubernetes from the cloud to the edge, enabling the management of edge infrastructure with a unified standard. Through the integration with KubeEdge, it solves the problems of edge node management, edge workload scheduling and edge observability, and extends the hybrid cloud management to the edge side with the existing multi-cluster management of KubeSphere.

In addition, V3.1.0 received more contributions and participation from enterprises and users outside of QingCloud, whether it was feature development, feature testing, defect reporting, requirements recommendations, enterprise best practices, Bug fixes, international translations, documentation contributions, These contributions from the open source community were instrumental in the release and promotion of V3.1.0, and we will give special thanks at the end of this article.

Interpret the major update v3.1.0

KubeSphere 3.1.0 adds the metering and billing function to support the metering and statistics of application resource consumption of multi-tier cluster and enterprise space and multi-tenant. By integrating KubeEdge, applications can be quickly distributed to edge nodes. It also provides more powerful observability capabilities, such as PromQL compatibility, built-in mainstream alarm rules, visual docking pins, enterprise wechat, Slack and Webhook notification channels. DevOps is also a step up in ease of use in 3.1.0, with several common pipeline templates built in, support for multi-branch pipelines and pipeline replication, etc. See the poster at the end of this article for details of major updates.

Multi-dimensional metering and charging make K8s operating costs more transparent

When an enterprise operates and manages the Kubernetes container platform, it is often necessary to analyze resource consumption, view the consumption bills of the cluster and its tenants, gain insight into resource usage, and analyze infrastructure operating costs.

In KubeSphere 3.1.0, platform resource consumption can be analyzed from several dimensions:

  • From the cluster dimension, you can view the resource consumption of each cluster, analyze the workload on each node, and accurately plan the resource usage of the workload on each node.
  • From the perspective of enterprise space, you can view the resource consumption of each enterprise space, obtain the consumption bills of projects, applications, and workloads in the enterprise space, and analyze the resource usage of each tenant in a multi-tenant environment.

In addition to the interface for viewing and exporting data, the KubeSphere metering platform also provides apis for all operations. In the following versions, we will continue to strengthen and build a complete end-to-end metering and billing operational system.

Edge Node Management

KubeEdge is an open source edge computing platform. Based on Kubernetes’ native container scheduling and scheduling capabilities, it implements cloud-side collaboration, computing sink, massive edge device management, edge autonomy and other capabilities. However, KubeEdge lacks the support of cloud control level. If KubeSphere and KubeEdge are combined, this problem can be solved well to realize unified distribution and management of applications and workloads in cloud and edge nodes.

This idea was realized in V3.1.0. KubeSphere now supports KubeEdge edge node management, KubeEdge cloud component installation and deployment, and edge node log and monitoring data collection and display. Combined with the edge autonomy function of KubeEdge and the multi-cloud and multi-cluster management function of KubeSphere, the cloud-edge-end integrated management and control can be realized to solve the requirements of unified application delivery, operation and maintenance and control on a large number of edge and end devices.

We will strengthen our ability to manage micro-services

Based on Istio, KubeSphere provides traffic governance functions such as Canary publishing, blue-green deployment, and fuses. It also supports visualization of topological relationships between microservices and provides fine-grained monitoring data. In terms of distributed link tracing, KubeSphere uses Jaeger to allow users to quickly track communication between microservices, making it easier to understand request delays, performance bottlenecks, serialization and parallel calls to microservices.

KubeSphere 3.1.0 has enhanced the microservices governance feature, upgrading Istio to 1.6.10 and supporting graphical traffic direction detection, which shows the incoming/outgoing application traffic in a graphical manner. In addition, the Nginx Ingress Gateway can be monitored, and the Nginx Ingress Controller can be monitored.

Multi-cloud and multi-cluster management

Although multi-cloud and multi-cluster management with KubeSphere 3.0.0 provides a central control panel for multiple Kubernetes clusters, enabling deployment, operation and maintenance of applications across clouds and clusters, However, the Member cluster management service relies on components such as Redis, OpenLDAP, and Prometheus and is not suitable for lightweight deployment. KubeSphere 3.1.0 removes these dependencies, makes the Member Cluster Management service more lightweight, and reconstructs the cluster controller to support running Tower Agent services in a highly available manner.

Greater observability

Observability is a key part of container cloud platform. In narrow sense, observability mainly includes monitoring, logging and tracking, etc., and in broad sense, it also includes alarm, event and audit, etc. 3.1.0 Optimized and upgraded the existing monitoring, log, and alarm functions, and added more new features.

  • Monitoring: Supports configuration in graphical modeServiceMonitor, adding custom monitoring at the cluster level, and also implementing PromQL syntax highlighting similar to Grafana.
  • Warning: Architecture adjustment was made in V3.1.0. Instead of using MySQL, Redis, ETCD and other components as well as the old version of alarm rule format, Thanos Ruler is used for alarm management in conjunction with Prometheus built-in alarm rules and compatible with Prometheus alarm rules.
  • Notification management: Architecture adjustment is completed, and comprehensive integration with self-developed Notification Manager V1.0.0 is achieved to connect Notification channels such as email, Dingding, enterprise wechat, Slack and Webhook with graphical interface.
  • Logging: Added support for Loki to export logs to Loki. Also added to kubelet docker/containerd log collection.

Easier DevOps

KubeSphere 3.1.0 added GitLab multi-branch pipeline and pipeline cloning functions, and built-in common pipeline templates to help DevOps engineers improve the efficiency of CI/CD pipeline creation and operation. Most scenarios can be modified based on pipelined templates, eliminating the need to create from scratch and achieving true out-of-the-box.

A flexible and pluggable cluster installation tool

KubeKey not only supports Kubernetes 1.17 ~ 1.20 on AMD 64 and ARM 64, but also supports K3s. In addition, Kubekey also supports Cilium, Kube-OVn and other network plug-ins. Since Dockershim was deprecated in K8s 1.20, Kubekey can be used to deploy containerd, Cri-O, iSula and other container runtimes to allow users to quickly create clusters on demand.

Operation and maintenance friendly network management

KubeSphere inherits the network power of IaaS cloud platform to container cloud platform, giving users the same stable, secure and easy-to-use network experience on Kubernetes as IaaS. V3.1.0 added a visual network topology that allows you to see the network call relationship between services.

Since Calico is one of the most commonly used Kubernetes CNI plug-ins today, V3.1.0 now supports Calico IP pool management and can also specify static IP for Deployment. In addition, v3.1.0 also added support for the Kube-OVN plug-in.

Authentication and multi-tenant

Unified identity management and complete authentication system are indispensable capabilities for logical isolation in multi-tenant systems. Based on Kubernetes role Access Control (RBAC), KubeSphere provides fine-grained permission control capabilities. In addition to supporting Namespace level resource isolation, it further defines tenants through Workspace. Multi-level permission control is more suitable for enterprise users.

V3.1.0 added the organizational structure management function to simplify batch authorization operations by user groups. In addition, it also supports the resource quota management of enterprise space to control the resource consumption and further meet the actual requirements of enterprise users.

For unified authentication, v3.1.0 simplifies the configuration of IdentityProvider (IdP). In addition to LDAP, it supports universal Authentication protocols such as Central Authentication Service (CAS), OpenID Connect (OIDC), and OAuth2, and provides a plug-in expansion mode to facilitate the integration between different account systems.

Full open source: Community vs. internationalization

Thanks to the power of the open source community, KubeSphere has rapidly gone global, with more than 10 million downloads in more than 90 countries and regions. V3.1.0 Console supports Chinese, English, Chinese and Spanish, and KubeSphere will further expand into overseas markets in the future.

KubeSphere 3.1.0 will continue to be 100% open source, and many of the new features introduced in 3.0.0 are already open on GitHub. For example, Porter, OpenPitrix, Fluentbit Operator, KubeKey, KubeEye, Notification Manager, Kube-Events. We also open source a set of front-end component library Kube Design. The code and design documentation for these new features can be found in the GitHub repository, and you are welcome to send us Star + Fork + PR on GitHub.

3.1.0 Major Updates at a glance

Install the upgrade

KubeSphere has synchronized and backed up all v3.1.0 images in the domestic mirror warehouse, so the installation experience of domestic users downloading images will be more friendly. For the latest v3.1.0 installation and upgrade guide, please refer to the official KubeSphere documentation.

Thank you

Here, in no particular order, are the GitHub ids of the major contributors to KubeSphere 3.1.0:


Kubernetes and Cloud Native Meetup Shanghai (May 15) is coming, come and join KubeShpere Community for an offline technology carnival in the early summer of Shanghai!

With the continuous development of KubeSphere community and the continuous increase of community users, KubeShpere ecology is gradually maturing. This Meetup, together with CNCF Foundation, we will invite experts from DevOps, database, storage and other industries to talk about their applications and practices in cloud native. There are more users from the community to share the way of containerization; The KubeSphere community will also be reading the latest features and Roadmap of KubeSphere 3.1.0. If you are interested, please scan the qr code below to register for free.

This article is published by OpenWrite!