Hi, I’m a fish fry, and in a real application, you do so many Server side, you write N RPC methods. Want to see the metrics of the method, but nowhere to start?

In this paper, gRPC + Opentracing + Zipkin will be used to build a distributed link tracking system to view the link and performance indicators of the whole system.

Opentracing

What is the

OpenTracing makes it easy for developers to add (or replace) tracking system implementations by providing platform-independent, vendor-independent apis

However, OpenTracing is not a standard. CNCF is not an official standards body, but its goal is to create more standard apis and tools for distributed tracking

Noun explanation

Trace

A trace represents the execution of a transaction or process in a (distributed) system

Span

A SPAN represents a single unit of work done in a distributed system. Also includes “references” to other spans, which allows you to compose multiple Spans into a complete Trace

Each SPAN encapsulates the following according to the OpenTracing specification:

  • The operation name
  • Start time and end time
  • key:value span Tags
  • key:value span Logs
  • SpanContext

Tags

Span tags can be understood as user-defined Span comments. Easy to query, filter, and understand trace data

Logs

Span logs Record Span time periods or events. Mainly used to capture log information for a particular Span and other debugging or information output for the application itself

SpanContext

SpanContext represents the state passed across process boundaries to child spans. Often used when creating context in tracing diagrams

Baggage Items

Baggage Items can be understood as an additional collection of data transferred in the trace global run

A case study

You can see the following in the figure:

  • Execution time context
  • Hierarchical relationships between services
  • Serial or parallel invocation chains between services

Combined with the above information, in the actual scene, we can find the pain point of the system by referring to the context of the whole system call chain, performance and other indicators

Zipkin

What is the

Zipkin is a distributed tracking system. Its role is to collect the timing data needed to address latency issues in the microservices architecture. It manages the collection and lookup of this data

Zipkin’s design is based on the Google Dapper paper.

run

docker run -d -p 9411:9411 openzipkin/zipkin
Copy the code

For other installation methods, see github.com/openzipkin/…

validation

Visit http://127.0.0.1:9411/zipkin/ to check if Zipkin running normally

gRPC + Opentracing + Zipkin

In the previous section, we did the following preparations:

  • What is Opentracing
  • Build Zipkin to provide distributed tracking system functions

Next, gRPC connects with Zipkin through Opentracing standard API, and then views the data through Zipkin

The directory structure

Create the simple_zipkin_client and simple_zipkin_server directories as follows:

Go - GRPC - example ├ ─ ─ LICENSE ├ ─ ─ the README. Md ├ ─ ─ client │ ├ ─ ─... │ ├ ─ ─ simple_zipkin_client ├ ─ ─ the conf ├ ─ ─ PKG ├ ─ ─ proto ├ ─ ─ server │ ├ ─ ─... │ ├─ ├─ Simple_zipkin_server ├─Copy the code

The installation

$ go get -u github.com/openzipkin/zipkin-go-opentracing
$ go get -u github.com/grpc-ecosystem/grpc-opentracing/go/otgrpc
Copy the code

gRPC

Server

package main

import (
	"context"
	"log"
	"net"

	"github.com/grpc-ecosystem/go-grpc-middleware"
	"github.com/grpc-ecosystem/grpc-opentracing/go/otgrpc"
	zipkin "github.com/openzipkin/zipkin-go-opentracing"
	"google.golang.org/grpc"

	"github.com/EDDYCJY/go-grpc-example/pkg/gtls"
	pb "github.com/EDDYCJY/go-grpc-example/proto"
)

type SearchService struct{}

func (s *SearchService) Search(ctx context.Context, r *pb.SearchRequest) (*pb.SearchResponse, error) {
	return &pb.SearchResponse{Response: r.GetRequest() + " Server"}, nil
}

const (
	PORT = "9005"

	SERVICE_NAME              = "simple_zipkin_server"
	ZIPKIN_HTTP_ENDPOINT      = "http://127.0.0.1:9411/api/v1/spans"
	ZIPKIN_RECORDER_HOST_PORT = "127.0.0.1:9000"
)

func main() {
	collector, err := zipkin.NewHTTPCollector(ZIPKIN_HTTP_ENDPOINT)
	iferr ! = nil { log.Fatalf("zipkin.NewHTTPCollector err: %v", err)
	}

	recorder := zipkin.NewRecorder(collector, true, ZIPKIN_RECORDER_HOST_PORT, SERVICE_NAME)

	tracer, err := zipkin.NewTracer(
		recorder, zipkin.ClientServerSameSpan(false),iferr ! = nil { log.Fatalf("zipkin.NewTracer err: %v", err)
	}

	tlsServer := gtls.Server{
		CaFile:   ".. /.. /conf/ca.pem",
		CertFile: ".. /.. /conf/server/server.pem",
		KeyFile:  ".. /.. /conf/server/server.key",
	}
	c, err := tlsServer.GetCredentialsByCA()
	iferr ! = nil { log.Fatalf("GetTLSCredentialsByCA err: %v", err)
	}

	opts := []grpc.ServerOption{
		grpc.Creds(c),
		grpc_middleware.WithUnaryServerChain(
			otgrpc.OpenTracingServerInterceptor(tracer, otgrpc.LogPayloads()),
		),
	}
    ...
}
Copy the code
  • Zipkin.newhttpcollector: Create a Zipkin HTTP back-end collector
  • Zipkin.newrecorder: Create a recorder based on the Zipkin collector
  • Zipkin. NewTracer: Create an OpenTracing tracker (compatible with Zipkin Tracer)
  • Otgrpc. OpenTracingClientInterceptor: Return GRPC UnaryServerInterceptor, difference is that the interceptors can be in GRPC Metadata find OpenTracing SpanContext. If found, the child node of the Span Context of the service
  • Otgrpc. LogPayloads: Sets and returns options. OpenTracing records application payloads in both directions.

Basically, you initialize the Zipkin, which in turn contains a collector, logger, and tracker. Then interceptor in the Server side to achieve SpanContext, Payload bidirectional read and management

Client

func main() {
	// the same as zipkin server
	// ...
	conn, err := grpc.Dial(":"+PORT, grpc.WithTransportCredentials(c),
		grpc.WithUnaryInterceptor(
			otgrpc.OpenTracingClientInterceptor(tracer, otgrpc.LogPayloads()),
		))
	...
}
Copy the code
  • Otgrpc. OpenTracingClientInterceptor: return GRPC. UnaryClientInterceptor. The core functions of the interceptor are:

(1) OpenTracing SpanContext injects gRPC Metadata

Create a ChildOf reference to create a child Span if there is a parent Span

Otherwise, as on the Server side, we initialize Zipkin first and then add interceptors that are specific to the Client side. We can finish the groundwork

validation

Start server. go and execute client. go. Check the schematic diagram of http://127.0.0.1:9411/zipkin/ :

complex

Here, try it out for yourself.

conclusion

In the multi-service architecture, serial, parallel, service is a very common situation, and it is often difficult to find the problem with conventional solutions (too much cost). This is where distributed tracking comes in handy

I hope you can understand its concept, build and apply a tracking system through the introduction and study of this chapter.

?

If you have any questions or mistakes, welcome to raise questions or give correction opinions on issues. If you like or are helpful to you, welcome Star, which is a kind of encouragement and promotion for the author.

My blog

Learn to fry fish Go: github.com/eddycjy/blo…

My official account

reference

Sample code for this series

  • go-grpc-example