There is an issue #9066 in Istio asking to replace the default Service Graph used in Istio with Kiali. Kiali was originally open sourced by Red Hat to address the issue of observability, or visibility of microservices, in the Service Mesh. It has received official support from the Istio community.

About Kiali

Individual applications are split into combinations of many microservices using the microservices architecture. As the number of services increases significantly, there is a need to understand communication patterns between services, such as fault-tolerance (through timeouts, retries, disconnections, and so on) and distributed tracing to be able to see where service invocations are going. The service grid provides these services at the platform level and frees application writers from the onerous communication patterns described above. Routing decisions are made at the grid level. Kiali works with Istio to visualize features such as service grid topologies, circuit breakers, and request rates. Kiali also includes Jaeger Tracing, which provides distributed Tracing out of the box.

Features provided by Kiali

Kiali provides the following features:

  • Service topology view
  • Distributed tracking
  • Metrics measure collections and ICONS
  • The configuration check
  • Health check and display
  • Service discovery

The following figure shows the service topology of the Bookinfo example shown in Kiali.

You can use Kubernetes-Vagrant-centos-cluster to quickly start a Kubernetes cluster running Kiali.

Build installation and trial

Kilia pod in the running process is/opt/kiali/kiali – config/kiali – configuration/config. The yaml – v 4.

/ kiali – configuration/config. The yaml is using ConfigMap mounted inside, used to configure kiali Web root and external service address.

server:
  port: 20001
  web_root: /
external_services:
  jaeger:
    url: "http://172.17.8.101:31888"
  grafana:
    url: "http://grafana.istio-system:3000"
Copy the code

Basic concepts in Kiali

Before we understand how Kilia provides microservice observability in Service Mesh, we need to understand how Kilia divides monitoring categories.

  • Application: To use the running workload, the Label must be marked as IstioappIn order to be. Note that if there are multiple versions of an application, justappTags with the same value belong to the same application.
  • Deployment: Deployment in Kubernetes.
  • Label: This value is important for Istio because Istio uses it to mark metrics. Each Application requirement includesapp 和 versionTwo label.
  • Namespace: Usually used to distinguish projects from users.
  • ServiceKubernetes Service (Kubernetes ServiceappThe label.
  • Workload: All common resource types in Kubernetes such as Deployment, StatefulSet, Job, and so on can be detected, whether or not these loads are added to the Istio Service Mesh.

The following figure shows the relationship between Application, Workload, and Service.

For details about Kilia API usage, please check the Swagger API documentation. Run the following command in Kiali’s root directory to check the API documentation.

make swagger-serve
Copy the code

Swagger UI is shown below.

architecture

After Kiali is deployed, only one Pod is started, and both the front and back ends are integrated into that one Pod. Kiali also has dependencies, such as the deployment of Prometheus in istio-system to obtain monitoring metrics in Kiali’s pages. Below is the architecture of Kiali, from the Kiali website.

Kiali uses a traditional back-end separation architecture:

  • The back end is written using Go: github.com/kiali/kiali, which provides apis for the front end, all messages are encoded using JSON, and ConfigMap and Secret are used to store configuration. Communicate directly with Kubernetes and Istio to get data.
  • The front end is written in Typescript: github.com/kiali/kiali… Stateless, except that some certificates are saved in the browser. You can jump to the Jaeger Distributed tracing and Grafana monitoring pages by querying the back-end API.

Jaeger and Grafana are optional components that use external services that are not deployed by Kiali and require urls to be configured in Kiali-configmap.yaml. Note that this URL must be directly accessible from your local browser.

Note: No data is saved in Prometheus and the service topology cannot be displayed if there are no requests between services, so people send requests to the ProductPage service to generate the service topology after deploying the Bookinfo service.

Service topology view

The service topology in Kiali is more polished and intuitive than ServiceGraph, which Istio originally deployed by default, with more options.

For example, using CURL to simulate a request.

$ curl -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VybmFtZSI6ImFkbWluIiwiZXhwIjoxNTM5NjczOTYyfQ.6gNz4W6yA9Bih4RkTbcSvqdaiRqsyj8c 8o6ictM9iDs" http://172.17.8.101:32439/api/namespaces/all/graph?duration=60s&graphType=versionedApp&injectServiceNodes=false&appenders=dead_node,sidecars_check,istio
Copy the code

The following JSON return value is returned, partially omitted to save space:

{
  "timestamp": 1539296648."graphType": "versionedApp"."elements": {
    "nodes": [{"data": {
          "id": "6519157be154675342fb76c41edc731c"."nodeType": "app"."namespace": "default"."app": "reviews"."isGroup": "version"}},... {"data": {
          "id": "6249668dd0a91adb9e62994d36563365"."nodeType": "app"."namespace": "istio-system"."workload": "istio-ingressgateway"."app": "istio-ingressgateway"."version": "unknown"."rateOut": "0.691"."isOutside": true."isRoot": true}}]."edges": [{"data": {
          "id": "d51ca2a95d721427bbe27ed209766ec5"."source": "06e488a37fc9aa5b0e0805db4f16ae69"."target": "31150e7e5adf85b63f22fbd8255803d7"."rate": "0.236"."percentRate": "17.089"."responseTime": "0.152"}},... {"data": {
          "id": "1dda06d9904bcf727d1b6a113be58556"."source": "80f71758099020586131c3565075935d"."target": "4b64bda48e5a3c7e50ab1c63836c9469"."rate": "0.236"."responseTime": "0.022"}}]}}Copy the code

This value contains the information of each node and eDEge. Node refers to each node in the figure, which contains the configuration information of nodes, Edge refers to the relationship between nodes and traffic status. The front end can draw a service topology from this information, so let’s take a look at kiali’s back end to see how it generates JSON information in the above format.

Note: For detailed REST API usage and field descriptions, see the API documentation generated by Swagger.

Code parsing

Here’s a look at the basic structure of Kiali’s back-end code.

The routing configuration

The route information of the service topology is saved in the kiali/routing/routes.go file.

{
	"GraphNamespace",
	"GET",
	"/api/namespaces/{namespace}/graph",
	handlers.GraphNamespace,
	true,
},
{
	"GraphAppVersion",
	"GET",
	"/api/namespaces/{namespace}/applications/{app}/versions/{version}/graph",
	handlers.GraphNode,
	true,
},
{
	"GraphApp",
	"GET",
	"/api/namespaces/{namespace}/applications/{app}/graph",
	handlers.GraphNode,
	true,
},
{
	"GraphService",
	"GET",
	"/api/namespaces/{namespace}/services/{service}/graph",
	handlers.GraphNode,
	true,
},
{
	"GraphWorkload",
	"GET",
	"/api/namespaces/{namespace}/workloads/{workload}/graph",
	handlers.GraphNode,
	true,
}
Copy the code

Looking directly at the API documentation generated by Swagger also works.

PQL query statement construction

Handlers /graph.go processes HTTP requests, and all indicator information in the service topology is obtained from Prometheus.

The Kiali service status topology is queried according to the namespace. For example, the service indicator query PQL in the default namespace is as follows:

round(sum(rate(istio_requests_total{reporter="source",source_workload_namespace="default",response_code=~"[2345] [0-9] [0-9]." "} [600s])) by (source_workload_namespace,source_workload,source_app,source_version,destination_service_namespace,destination_service_n Ame, destination_workload destination_app destination_version, response_code), 0.001)Copy the code

One of the parameters are incoming through page selection (build the option in the PQL in kiali/graph/options/options. Go) defined in:

  • reporter="source": metric Indicates the source of the reportenvoyThe downstream client of the agent. inService gridIn, a source service is usually oneThe workloadHowever, the source service that imports traffic may contain other clients, such as a browser, or a mobile application.
  • source_workload_namespace="default": Select a namespace.
  • response_code: Returns to the code area.
  • [600s]: Indicates the interval in the queried data.

For details about how to use PQL, see QUERY EXAMPLES – Prometheus. IO.

This information contains all workload traffic information. You can perform simple operations to calculate application/ Service traffic.

HTTP processing logic

The logical entry to the HTTP request is located at kiali/ Handlers /graph.go.

func graphNamespaces(o options.Options, client *prometheus.Client) graph.TrafficMap {
	switch o.Vendor {
	case "cytoscape":
	default:
		checkError(errors.New(fmt.Sprintf("Vendor [%s] not supported", o.Vendor)))
	}

	log.Debugf("Build [%s] graph for [%v] namespaces [%s]", o.GraphType, len(o.Namespaces), o.Namespaces)

	trafficMap := graph.NewTrafficMap()
	for _, namespace := range o.Namespaces {
		log.Debugf("Build traffic map for namespace [%s]", namespace)
		namespaceTrafficMap := buildNamespaceTrafficMap(namespace, o, client)

		for _, a := range o.Appenders {
			a.AppendGraph(namespaceTrafficMap, namespace) // Appender is used to add service Graph
		}
		mergeTrafficMaps(trafficMap, namespaceTrafficMap) // Merge the service states of different namespaces
	}

	// Appender is used to add/delete/modify node information. After the operation is complete, you can determine the following:
	// - mark it as an outsider (i.e. a node not in the requested namespace)
	// - marks it as an internal traffic producer (i.e., edge only facing out in namespace)
	markOutsiders(trafficMap, o)
	markTrafficGenerators(trafficMap)

	if graph.GraphTypeService == o.GraphType {
		trafficMap = reduceToServiceGraph(trafficMap)
	}

	return trafficMap
}
Copy the code

Appender is an interface that injects detailed information into the Service Graph. It is defined as follows:

// The Appender is implemented by any code provided to attach the Service Graph with supplementary information. If something goes wrong, the appender should execute Panic and treat it as an error response.
type Appender interface {
	AppendGraph performs appender work on the provided traffic map. The Map may initially be empty. Allows the appender to add or remove mapping entries.
	AppendGraph(trafficMap graph.TrafficMap, namespace string)}Copy the code

Appender is located in the Kiali /graph/ Appender directory and currently has the following implementations:

  • DeadNodeAppender: Used to remove unwanted nodes from the Service Graph.
  • IstioAppender: Obtains details about Istio of a specified namespace. The current version obtains VirtualService and DestinationRule information of a specified namespace.
  • ResponseTimeAppender: Gets the response time.
  • SecurityPolicyAppender: Adds security policy information to the Service Graph.
  • SidecarsCheckAppender: Checks Sidecar configuration information, such as whether App Label exists in Pod.
  • UnusedNodeAppender: node that is not appended to the Service Mesh.

Let’s take a look at the TrafficMap structure defined in Kiali /graph/graph.go.

// TrafficMap is a mapping between apps and nodes, each of which optionally holds Edge data. Metadata is a generic mapping used to store any desired Node or edge information. Each APP node should have a unique namespace + workload. Note that it is possible but probably not common to have two nodes with the same name + version in the same namespace.
type TrafficMap map[string]*Node

type Node struct {
	ID        string                 // unique identifier for the node
	NodeType  string                 // Node type
	Namespace string                 // Namespace
	Workload  string                 // Workload (deployment) name
	App       string                 // Workload app label value
	Version   string                 // Workload version label value
	Service   string                 // Service name
	Edges     []*Edge                // child nodes
	Metadata  map[string]interface{} // app-specific data
}

type Edge struct {
	Source   *Node
	Dest     *Node
	Metadata map[string]interface{} // app-specific data
}
Copy the code

The above is just the interpretation of Kiali part of the code, more detailed implementation we can clone Kiali code for their own research.

reference

  • Kiali.io
  • QUERY EXAMPLES – prometheus.io
  • replace Service Graph with Kiali #9066 – github.com
  • rootsongjc/kubernetes-vagrant-centos-cluster – github.com

ServiceMesher community information

Wechat group: Contact me to join the group

Community official website: www.servicemesher.com

Slack:servicemesher.slack.com requires invitation to join

Twitter: twitter.com/servicemesh…

GitHub：github.com/

servicemesher

For more ServiceMesh consultation, follow the wechat public account ServiceMesher.