Moment For Technology

Embrace Agent, "0" code to play Trace OpenTelemetry series second bomb!

Posted on Aug. 8, 2022, 7:08 p.m. by Millie Forbes-Todd
Category: The back-end Tag: The back-end Micro service


OpenTelemetry is compatible with OpenTracing and has many conceptual similarities. OpenTracing is a bit like The present tense of OpenTelemetry, which is the future tense of OpenTracing. Since OpenTelemetry includes Tracing, Logging, and Metrics and is still in the SandBox stage, we will first explore this area through OpenTracing.

The authors introduce

Xu for

Tencent Cloud Micro service team senior solution architect

He graduated from Erasmus Mundus IMMIT with a Master's degree in Economics and IT Management

Since 2006 he has worked as software development engineer, data development engineer and solution architect at SonyEricsson, SAP, Cloud and other companies

Firstly, from the view of data structure of Tracing, OpenTracing and OpenTelemetry are very similar. Here I will start with an example of the Structure of OpenTracing (since OpenTelemetry is committed to compatibility with OpenTracing, plus OpenTelemetry is currently a SandBox project and there may be many changes).

Tracing basic data structures

As shown in the figure below, a single Tracing link can be represented by a structure similar to a tree. At the top is the initiator of the link, and below are all the spans occurring under this Trace. Span can be parent-child logic (as shown in the relationship between SpanB and SpanC to SpanA, which can be executed sequentially or in parallel), or logic that follows sfrom (as in SpanG and SpanF).

\# ref: Causal Relationships between Spans in a single Trace \ [Span a \] please please please (the root Span) | + -- -- -- -- -- - + -- -- -- -- -- -- + | | \ [Span B \] \ [C \] Span Please please please (Span C is a \ ` ChildOf \ ` Span a) | | \ [\] Span D + + -- -- -- -- -- -- -- -- -- -- + | | \ [Span E \] \ [Span F \]    \ [Span G \]    \ [Span H \] ↑ : (Span G \ 'FollowsFrom\' Span F)Copy the code

To be more specific, FollowsFrom means that the parent Span does not wait for the child Span to return. Many scenarios follow this logic, such as waiting for the merchant to confirm a successful order - the merchant confirmation process is triggered at the time of the order, but does not need the merchant confirmation to complete the order. There are many similar scenarios, but there is no more detailed classification in OpenTracing.

If you look at the timeline, it looks something like this, where each service starts and ends, and the relationship between the top and bottom is displayed in detail on the Web GUI.

And then talk specifically about how those spans are created and what information is required upstream and downstream. Many of the components are integrated into the framework, providing a free lunch but also blocking the creation of data structures, so let's take a look at the creation process using the basic OpenTracing SDK. Here is the simplest Hello World program, with the simplest code to complete a whole process.

\# Ref: const http = require('http'); const opentracing = require('opentracing'); ## here is the top quote // NOTE: the default OpenTracing tracer does not record any tracing information. // Replace this line with the tracer implementation of your choice. const tracer = new opentracing.Tracer(); Const Span = tracer.startSPAN (' HTTP \_request'); const Span = tracer.startspan (' HTTP \_request'); ## Span const opts = {host: '', method: 'GET', port: '80', path: '/',}; http.request(opts, res = { res.setEncoding('utf8'); res.on('error', err = { // assuming no retries, mark the span as failed span.setTag(opentracing.Tags.ERROR, true); Log ({'event': 'error', 'error.object': err, 'message': err.message, 'stack': err.stack}); Log Span. Finish (); }); res.on('data', chunk = { span.log({'event': 'data\_received', 'chunk\_length': chunk.length}); }); res.on('end', () = { span.log({'event': 'request\_end'}); span.finish(); }); }).end();Copy the code

Then the customer will ask: ** How does the service get the information (TraceID) and initiate the next Span? ** This leads to the following code that parses Ctx and opens a new Span.

// Use the inbound HTTP request's headers as a text map carrier. var headersCarrier = inboundHTTPReq.headers; var wireCtx = Tracer.extract(Tracer.FORMAT\_HTTP\_HEADERS, headersCarrier); var serverSpan = Tracer.startSpan('... ', { childOf : wireCtx });Copy the code

** What about those FollowsFrom? You don't get the chance to read messages from Http Headers, which are stored in standard Kafka components. ** This can not be difficult to industry leaders, see the code below two paragraphs of comment, perfect interpretation of how to pass information.

// from // Register tracer with GlobalTracer: GlobalTracer.register(tracer); // Add TracingProducerInterceptor to sender properties: senderProps.put(ProducerConfig.INTERCEPTOR\_CLASSES\_CONFIG, TracingProducerInterceptor.class.getName()); // Instantiate KafkaProducer KafkaProducerInteger, String producer = new KafkaProducer(senderProps); // Send producer.send(...) ; // Add TracingConsumerInterceptor to consumer properties: consumerProps.put(ConsumerConfig.INTERCEPTOR\_CLASSES\_CONFIG, TracingConsumerInterceptor.class.getName()); // Instantiate KafkaConsumer KafkaConsumerInteger, String consumer = new KafkaConsumer(consumerProps); //Subscribe consumer.subscribe(Collections.singletonList("messages")); // Get records ConsumerRecordsInteger, String records = consumer.poll(1000); // To retrieve SpanContext from polled record (Consumer side) ConsumerRecordInteger, String record = ... SpanContext spanContext = TracingKafkaUtils.extractSpanContext(record.headers(), tracer);Copy the code

The Interceptor creates a Span when sending a Producer and inserts the SpanContext into the Record Headers.

  // Class TracingKafkaUtils  
  public static void inject(SpanContext spanContext, Headers headers,
      Tracer tracer) {
    tracer.inject(spanContext, Format.Builtin.TEXT\_MAP, new HeadersMapInjectAdapter(headers));
Copy the code

The Span is then closed when the Consumer reads, and the cycle is complete.

  // TracingConsumerInterceptorK, V implements ConsumerInterceptorK, V
  public ConsumerRecordsK, V onConsume(ConsumerRecordsK, V records) {
    for (ConsumerRecordK, V record : records) {
      TracingKafkaUtils.buildAndFinishChildSpan(record, GlobalTracer.get());

    return records;
Copy the code

If there are children ask: ** that.. What about the other components? What about MQ like Pulsar? The conclusion is the same. Most open source components have long recognized the importance of Tracing and offer similar functionality. For example, pulsar-Tracing, a free lunch integrated with OpenTracing, was specially developed for PulSAR.

// // Instantiate Producer with tracing interceptor. ProducerString producer = client .newProducer(Schema.STRING) .intercept(new TracingProducerInterceptor()) .topic("your-topic") .create(); // Send messages. producer.send("Hello OpenTracing!" ); // Instantiate Consumer with tracing interceptor. ConsumerString consumer = client.newConsumer(Schema.STRING) .topic("your-topic") .intercept(new TracingConsumerInterceptor()) .subscriptionName("your-sub") .subscribe(); // Receive messages. MessageString message = consumer.receive(); // To retrieve SpanContext from the message(Consumer side). SpanContext spanContext = TracingPulsarUtils.extractSpanContext(message, tracer);Copy the code

If the logic of how OpenTracing delivers information is clear, what is the last Span to display in the interface?

As shown in the figure below, various information is provided for us to analyze and deal with when we encounter problems. Most of the information that has been deposited over many years is found to be necessary to solve the problem. These attributes are adopted and integrated by open source projects. If there are some that haven't been integrated yet, you are welcome to contribute.

Then there's the raw information

There are three types of spans (Root Span, Child Span and FollowsFrom). The following code shows how to create the three forms of spans (demo code javascript).

// Start a new (parentless) root Span:
var parent = Tracer.startSpan('DoWork');

// Start a new (child) Span:
var child = Tracer.startSpan('load-from-db', {
    childOf: parent.context(),

// Start a new async (FollowsFrom) Span:
var child = Tracer.startSpan('async-cache-write', {
    references: \[
Copy the code

The code is pretty simple, the only thing that needs to be explained is the context part. So the context here is the SpanContext in the Span. This component holds SpanID and TraceID, which can string together the entire call chain.

With that in mind, you should be able to start Tracing your own microservices. Oh, no! Some languages do not need to write code, such as Java, Nodejs, and Python. All of them use the agent's ability to listen to Runtime and automatically generate Tracing logic.

Let me sort out some of the things that do and don't code:


Java passes the javaAgent argument, while NodeJS and Python need to start an additional process as the Agent. If you are interested, click into the document and have a look. It's very interesting.

\# e.g. skywalking python agent
from skywalking import agent, config

config.init(collector='XXX.X.X.X:XXXXX', service='your awesome service')
Copy the code


Zipkin. IO/pages/trace...

At present, the mainstream access method of Zipkin is to integrate the SDK of language and framework. There is no agent explicitly supported on the official website, while Jaegertracing provides a standard Agent component after version 0.8. As a non-invasive standard solution. When you need to use it and when you don't, you can see the documentation on the official website:

Generally speaking, agent avoids setting tracing related configuration in the program, avoids generating too many network links, and can capture some environment information (zone/region, etc.) independently. All these are reasons for choosing Agent instead of writing codes.

Suddenly not writing code feels decent?

Phase to recommend

"Today we talk about the Trace OpenTelemetry And TSW | overview"

Exclusive: Information Replication between Kafka clusters is here!

The Java is dead? Tencent said, "no!" | tencent itself Kona JDK technology share salon in registration"

About (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.