preface

KubeCube (Kubecube.io) is a lightweight enterprise container platform developed by NetEase, which provides enterprises with kubernetes resource visual management and unified multi-cluster multi-tenant management functions. The KubeCube community will use a series of technical articles to explain the design features and technical implementation of KubeCube to help developers and users quickly understand and use KubeCube. This article is the third part, focusing on the implementation of KubeCube user management and identity authentication.

User management

All Kubernetes clusters have two types of users: service accounts managed by Kubernetes and regular users.

Kubernetes assumes that ordinary users are managed by a cluster-independent service in one of the following ways:

  • The administrator responsible for distributing the private key
  • User databases like Keystone or Google Accounts
  • A file containing a list of user names and passwords

In view of this,

Kubernetes does not contain objects that represent ordinary user accounts

. Common user information cannot be added to the cluster through API calls.

Kubernetes does not directly provide user management features, does not support ordinary user objects, and does not store any information about ordinary users. To create a user, you need to create a private key and a certificate for the user to authenticate the user. In addition, because no user information is stored, the cluster administrator cannot centrally manage users and is unaware of other users. Therefore, KubeCube first redefine the concept of the User, that is, to provide the User resource type, storage User information, User management, at the same time convenient subsequent identity authentication and permission verification.

ApiVersion: user.kubecube. IO /v1 kind: user metadata: name: unique user id, user-defined, non-repeatable, and unmodifiable spec: password: DisplayName: user name Email: email address phone: phone number Language: language: en/ch loginType: User login mode: Normal/LDAP/Github /... State: user status: normal/forbidden status: lastLoginTime: lastLoginTime lastLoginIp: lastLoginIp addressCopy the code

Users can be manually created by the administrator on the front page or automatically created by the system upon the first login using external authentication. Therefore, users can be divided into common registered users and third-party authorized login users in terms of registration mode. However, in both cases, the corresponding User CR is created in the managed cluster. Warden’s resource synchronization manager then synchronizes the CR from the management cluster to the computing cluster for subsequent unified authentication.

In this way, you only need to query User resources in the managed cluster on the User management page to manage users in a centralized manner. In addition, you can easily add users, query users, and modify user meta-information.

The identity authentication

In KubeCube, local authentication and external authentication are supported. Local authentication means that an ordinary user is created in KubeCube, and then the user logs in and authenticates using the username and password registered at the time of creation. External authentication means that users can access KubeCube by authenticating their identities through third-party authentication platforms without creating users. The implementation of the two authentication methods will be described below.

The local certification

In KubeCube, users are authenticated mainly through JWT (JSON Web Token).

To log in to the system using local authentication, a user needs to enter the user name and password. KubeCube will query the User cr in the cluster based on the User name and compare the password. If the User cr is queried and the password is the same, the login succeeds. After updating a user’s login status, KubeCube generates a JWT based on the user name and splices it into a Bearer Token, which is stored and returned in a cookie.

The code for verifying the user name and password is as follows:

// generate token and return authJwtImpl := jwt.GetAuthJwtImpl() token, err := authJwtImpl.GenerateToken(&v1beta1.UserInfo{Username: name}) if err ! = nil { response.FailReturn(c, errcode.AuthenticateError) return } bearerToken := jwt.BearerTokenPrefix + " " + token c.SetCookie(constants.AuthorizationHeader, bearerToken, int(authJwtImpl.TokenExpireDuration), "/", "", false, true) user.Spec.Password = "" response.SuccessReturn(c, user) returnCopy the code

After the user successfully logs in, the front-end will bring the JWT with it for each subsequent request through cookies, and the back-end authentication middleware will verify the JWT. If valid, a new token is generated and returned, repeating the above process. Thus, even though the JWT generated in KubeCube is valid for 1 hour by default, the JWT is constantly refreshed to keep the user logged in as long as the user continues to visit.

Part of the code for authentication middleware is as follows:

func Auth() gin.HandlerFunc { return func(c *gin.Context) { if ! withinWhiteList(c.Request.URL, c.Request.Method, whiteList) { authJwtImpl := jwt.GetAuthJwtImpl() userToken, err := token.GetTokenFromReq(c.Request) if err ! = nil { response.FailReturn(c, errcode.AuthenticateError) return } newToken, respInfo := authJwtImpl.RefreshToken(userToken) if respInfo ! = nil { response.FailReturn(c, errcode.AuthenticateError) return } v := jwt.BearerTokenPrefix + " " + newToken c.Request.Header.Set(constants.AuthorizationHeader, v) c.SetCookie(constants.AuthorizationHeader, v, int(authJwtImpl.TokenExpireDuration), "/", "", false, true) c.Next() } } }Copy the code

External certification

At present, the implementation of external authentication is mainly divided into three kinds, namely general authentication, LDAP authentication and OAuth2 authentication.

The common

To make it easier for users to connect to their own authentication system, KubeCube supports a universal authentication mode. Users can enable the universal authentication mode and configure the address of the authentication system, so that users will go to their own authentication system for authentication every time they access KubeCube. After passing the authentication, the user name needs to be returned to KubeCube. KubeCube will still generate the corresponding Bearer Token based on the user name and place it in the header for subsequent permission verification. The main logical code implementation is as follows:

func Auth() gin.HandlerFunc { return func(c *gin.Context) { if ! withinWhiteList(c.Request.URL, c.Request.Method, whiteList) { authJwtImpl := jwt.GetAuthJwtImpl() if generic.Config.GenericAuthIsEnable { h := generic.GetProvider() user, err := h.Authenticate(c.Request.Header) if err ! = nil || user == nil { clog.Error("generic auth error: %v", err) response.FailReturn(c, errcode.AuthenticateError) return } newToken, error := authJwtImpl.GenerateToken(&v1beta1.UserInfo{Username: user.GetUserName()}) if error ! = nil { response.FailReturn(c, errcode.AuthenticateError) return } b := jwt.BearerTokenPrefix + " " + newToken c.Request.Header.Set(constants.AuthorizationHeader, b) } c.Next() } } }Copy the code

LDAP authentication

When a user selects the LDAP login mode, the user enters the user name and password. The system checks whether the user exists in the cluster and whether the user is in the Disabled state. If not, LDAP authentication starts:

  1. As an LDAP client, KubeCube obtains the user name and password and sends the administrator binding request packet to the LDAP server using the administrator DN and password as parameters to obtain the query permission.

  2. After receiving the binding request packet, the LDAP server verifies the administrator DN and password. If the administrator DN and password are correct, a binding response packet is sent to KubeCube.

  3. After receiving the binding response packet, KubeCube constructs filtering conditions based on the user name parameters entered by the user and sends the user DN query request packet to the LDAP server. For example, set the filtering condition to CN=User2.

  4. After receiving the user DN query request packet, the LDAP server searches the user DN based on the query start point, query range, and filtering conditions specified in the packet. If the query succeeds, a response message is sent to KubeCube. The user DN can be one or more. If no one user is obtained, the user name or password is incorrect and the authentication fails.

  5. KubeCube sends a user binding request packet to the LDAP server based on the user DN and password entered by the user.

  6. After receiving the binding request packet, the LDAP server checks whether the password entered by the user is correct.

  • If the password entered by the user is correct, a successful binding response message is sent to KubeCube.
  • If the user enters an incorrect password, a binding failure response message is sent to KubeCube. KubeCube continues to send binding requests to the LDAP server using the next user DN as the parameter until one DN is successfully bound. If all user DN bindings fail, KubeCube notifies the user that authentication failed.

If the User does not exist in the cluster, the User CR is created based on the User name. The corresponding Bearer Token is generated based on the user name and stored in cookies to be carried in the next request to identify the user.

OAuth2 authentication

In KubeCube OAuth2 authentication uses authorization code mode, because this mode is the most complete function, the most rigorous process authorization mode. The usual authentication process for OAuth2 is:

  1. The user accesses the client, which directs the former to the authentication server.
  2. The user selects whether to authorize the client.
  3. Assuming that the user has granted authorization, the authentication server directs the user to the redirection URI (URI) specified by the client, along with an authorization code.
  4. The client receives the authorization code, along with the earlier “redirect URI,” and requests a token from the authentication server. This step is done on the server in the background of the client and is not visible to the user.
  5. After verifying the authorization code and redirection URI, the authentication server sends the Access token and refresh token to the client.

In KubeCube implementation, GitHub login as an example:

  1. When a user selects GitHub for authentication, the front-end forwards the request to GitHub.
  2. GitHub asks users if they agree to license to KubeCube;
  3. If the user agrees, GitHub redirects back to KubeCube (/oauth/redirect) and send back an authorization code (code);
  4. KubeCube uses the authorization code to request a token from GitHub (access_token);
  5. GitHub return token (access_token);
  6. KubeCube uses tokens (access_token), request user information data from GitHub;
  7. Query the cluster. If the User does not exist, create User CR based on the User information.
  8. Generate Bearer tokens for accessing clusters based on user names and return authentication success;
  9. The front end stores Bearer tokens in cookies that are carried on the next request.

OpenAPI certification

Based on the above design scheme, it can be easily inferred that the certification implementation of OpenAPI is also completed through JWT. User is bound to each group of AK and SK, and the corresponding User is queried through AK and SK, and then Bearer Token returns are generated through the user. Name. In the next request, the user needs to carry the Token in the Cookie or header, and the KubeCube authentication middleware can resolve the user’s identity through the Token, so as to complete authentication.

Cluster certification

After authentication, the middleware will refresh the token. However, if the token is directly carried in the request header to request Kube-Apiserver to complete cluster authentication, the authentication backend of Kube-Apiserver needs to be modified when KubeCube is deployed. To modify kube-Apiserver configuration. This would invade the native Kubernetes cluster and significantly increase KubeCube’s deployment and maintenance costs. Therefore, we need to build another module to help complete cluster authentication — Auth-Proxy.

When users access KubeCube and request Kubernetes resources, they will go to auth-proxy module after authentication middleware enters the pass-through interface. Auth-proxy resolved the Bearer Token in request to the corresponding User; Impersonation of the User impersonation is used to impersonate the request agent to Kube-Apiserver, impersonating the current User to request Kube-Apiserver to “bypass” authentication and facilitate subsequent authentication.

conclusion

KubeCube’s User management system is mainly based on User CRD. The authentication system supports two authentication modes: local authentication and external authentication. Local authentication is based on JWT. After external authentication passes the third-party authentication platform, a User CR needs to be created in the cluster for subsequent User management and permission binding. For cluster authentication, the Impersonation method provided by Kubernetes is primarily used. The overall design and implementation is relatively simple, adhering to the KubeCube lightweight design concept.

For more information see:

  • KubeCube website: www.kubecube.io/

  • KubeCube: github.com/kubecube-io…

  • An in-depth look at KubeCube multi-cluster management

  • KubeCube multi-level tenant model

  • KubeCube open Source: Six features to simplify the implementation of Kubernetes

  • NetEase shufan more open source projects

About the author: Jia Hui, senior engineer of NetEase Shufan, core member of KubeCube community