This is the 21st day of my participation in the November Gwen Challenge. Check out the event details: The last Gwen Challenge 2021

introduce

Manage logs properly in the Echo framework with a complete example.

What are the usage scenarios?

  • Automatic log scrolling
  • Split into multiple log files
  • Changing log Format
  • , etc.

We will use RK-boot to start the Echo framework microservice.

Please visit the following address for the full tutorial:

  • rkdocs.netlify.app/cn

The installation

go get github.com/rookie-ninja/rk-boot
Copy the code

Briefly describes the concept

Rk-boot uses the following two libraries to manage logs.

  • Zap manages log instances
  • Lumberjack Manages log rolling

Rk-boot defines two log types, which will be described in detail later, but will be briefly introduced here.

  • ZapLogger: standard log, used to record errors, Info, etc.
  • EventLogger: Records events, such as RPC requests, in JSON or Console format.

Quick start

In this example, we will try to change the path and format of the ZAP log.

1. Create the boot. Yaml

---
zapLogger:
  - name: zap-log                        # Required
    zap:
      encoding: json                     # Optional, options: console, json
      outputPaths: ["logs/zap.log"]      # Optional
echo:
  - name: greeter
    port: 8080
    enabled: true
Copy the code

2. Create a main. Go

Write a log to the zap-log instance.

package main

import (
	"context"
	"github.com/rookie-ninja/rk-boot"
	"github.com/rookie-ninja/rk-entry/entry"
)

func main(a) {
	// Create a new boot instance.
	boot := rkboot.NewBoot()

	// Bootstrap
	boot.Bootstrap(context.Background())

	// Write zap log
	rkentry.GlobalAppCtx.GetZapLoggerEntry("zap-log").GetLogger().Info("This is zap-log")

	// Wait for shutdown sig
	boot.WaitForShutdownSig(context.Background())
}
Copy the code

4. Verify

Folder structure

├ ─ ─ the boot. Yaml ├ ─ ─. Mod ├ ─ ─. Sum ├ ─ ─ logs │ └ ─ ─ zap. Log └ ─ ─ main. GoCopy the code

Log output

{"level":"INFO"."ts":"The 2021-10-21 T02:10:09. 279 + 0800"."msg":"This is zap-log"}
Copy the code

Configuration EventLogger

In the example above, we configured the ZAP log, this time we modify the EventLogger.

1. Create the boot. Yaml

---
eventLogger:
  - name: event-log                      # Required
    encoding: json                       # Optional, options: console, json
    outputPaths: ["logs/event.log"]      # Optional
echo:
  - name: greeter
    port: 8080
    enabled: true
Copy the code

2. Create a main. Go

Write logs to an event-log instance.

package main

import (
	"context"
	"github.com/rookie-ninja/rk-boot"
	"github.com/rookie-ninja/rk-entry/entry"
)

func main(a) {
	// Create a new boot instance.
	boot := rkboot.NewBoot()

	// Bootstrap
	boot.Bootstrap(context.Background())

	// Write event log
	helper := rkentry.GlobalAppCtx.GetEventLoggerEntry("event-log").GetEventHelper()
	event := helper.Start("demo-event")
	event.AddPair("key"."value")
	helper.Finish(event)

	// Wait for shutdown sig
	boot.WaitForShutdownSig(context.Background())
}
Copy the code

3. Start the main. Go

$ go run main.go
Copy the code

4. Verify

Folder structure

├ ─ ─ the boot. Yaml ├ ─ ─. Mod ├ ─ ─. Sum ├ ─ ─ logs │ └ ─ ─ event. The log └ ─ ─ main. GoCopy the code

Log contents

{
    "endTime":"The 2021-11-27 T01:56:56. 001 + 0800"."startTime":"The 2021-11-27 T01:56:56. 001 + 0800"."elapsedNano":423."timezone":"CST"."ids": {"eventId":"70b034b8-27af-43ad-97a5-82c99292297d"
    },
    "app": {"appName":"echo-demo"."appVersion":"master-f948c90"."entryName":""."entryType":""
    },
    "env": {"arch":"amd64"."az":"*"."domain":"*"."hostname":"lark.local"."localIP":"10.8.0.2"."os":"darwin"."realm":"*"."region":"*"
    },
    "payloads": {},"error": {},"counters": {},"pairs": {"key":"value"
    },
    "timing": {},"remoteAddr":"localhost"."operation":"demo-event"."eventStatus":"Ended"."resCode":"OK"
}
Copy the code

concept

In the example above, we tried ZapLogger and EventLogger. Let’s take a look at how rK-boot is implemented and used.

architecture

ZapLoggerEntry

ZapLoggerEntry is a wrapper around a ZAP instance.

// ZapLoggerEntry contains bellow fields.
// 1: EntryName: Name of entry.
// 2: EntryType: Type of entry which is ZapLoggerEntryType.
// 3: EntryDescription: Description of ZapLoggerEntry.
// 4: Logger: zap.Logger which was initialized at the beginning.
// 5: LoggerConfig: zap.Logger config which was initialized at the beginning which is not accessible after initialization..
// 6: LumberjackConfig: lumberjack.Logger which was initialized at the beginning.
type ZapLoggerEntry struct {
	EntryName        string             `yaml:"entryName" json:"entryName"`
	EntryType        string             `yaml:"entryType" json:"entryType"`
	EntryDescription string             `yaml:"entryDescription" json:"entryDescription"`
	Logger           *zap.Logger        `yaml:"-" json:"-"`
	LoggerConfig     *zap.Config        `yaml:"zapConfig" json:"zapConfig"`
	LumberjackConfig *lumberjack.Logger `yaml:"lumberjackConfig" json:"lumberjackConfig"`
}
Copy the code

How to configure ZapLoggerEntry in boot.yaml?

ZapLoggerEntry is fully compatible with THE YAML architecture of ZAP and LumberJack. Users can configure multiple instances of ZapLogger as required and access them by name.

Complete configuration:

---
zapLogger:
  - name: zap-logger                      # Required
    description: "Description of entry"   # Optional
    zap:
      level: info                         # Optional, default: info, options: [debug, DEBUG, info, INFO, warn, WARN, dpanic, DPANIC, panic, PANIC, fatal, FATAL]
      development: true                   # Optional, default: true
      disableCaller: false                # Optional, default: false
      disableStacktrace: true             # Optional, default: true
      sampling:                           # Optional, default: empty map
        initial: 0
        thereafter: 0
      encoding: console                   # Optional, default: "console", options: [console, json]
      encoderConfig:
        messageKey: "msg"                 # Optional, default: "msg"
        levelKey: "level"                 # Optional, default: "level"
        timeKey: "ts"                     # Optional, default: "ts"
        nameKey: "logger"                 # Optional, default: "logger"
        callerKey: "caller"               # Optional, default: "caller"
        functionKey: ""                   # Optional, default: ""
        stacktraceKey: "stacktrace"       # Optional, default: "stacktrace"
        lineEnding: "\n"                  # Optional, default: "\n"
        levelEncoder: "capitalColor"      # Optional, default: "capitalColor", options: [capital, capitalColor, color, lowercase]
        timeEncoder: "iso8601"            # Optional, default: "iso8601", options: [rfc3339nano, RFC3339Nano, rfc3339, RFC3339, iso8601, ISO8601, millis, nanos]
        durationEncoder: "string"         # Optional, default: "string", options: [string, nanos, ms]
        callerEncoder: ""                 # Optional, default: ""
        nameEncoder: ""                   # Optional, default: ""
        consoleSeparator: ""              # Optional, default: ""
      outputPaths: [ "stdout" ]           # Optional, default: ["stdout"], stdout would be replaced if specified
      errorOutputPaths: [ "stderr" ]      # Optional, default: ["stderr"], stderr would be replaced if specified
      initialFields:                      # Optional, default: empty map
        key: "value"
    lumberjack:                           # Optional
      filename: "rkapp-event.log"         # Optional, default: It uses <processname>-lumberjack.log in os.TempDir() if empty.
      maxsize: 1024                       # Optional, default: 1024 (MB)
      maxage: 7                           # Optional, default: 7 (days)
      maxbackups: 3                       # Optional, default: 3 (days)
      localtime: true                     # Optional, default: true
      compress: true                      # Optional, default: true
Copy the code

How to get ZapLogger in code?

Access by name.

// Access entry
rkentry.GlobalAppCtx.GetZapLoggerEntry("zap-logger")

// Access zap logger
rkentry.GlobalAppCtx.GetZapLoggerEntry("zap-logger").GetLogger()

// Access zap logger config
rkentry.GlobalAppCtx.GetZapLoggerEntry("zap-logger").GetLoggerConfig()

// Access lumberjack config
rkentry.GlobalAppCtx.GetZapLoggerEntry("zap-logger").GetLumberjackConfig()
Copy the code

EventLoggerEntry

Rk-boot treats each RPC request as an Event and logs it using the Event type in RK-Query.

// EventLoggerEntry contains bellow fields.
// 1: EntryName: Name of entry.
// 2: EntryType: Type of entry which is EventLoggerEntryType.
// 3: EntryDescription: Description of EventLoggerEntry.
// 4: EventFactory: rkquery.EventFactory was initialized at the beginning.
// 5: EventHelper: rkquery.EventHelper was initialized at the beginning.
// 6: LoggerConfig: zap.Config which was initialized at the beginning which is not accessible after initialization.
// 7: LumberjackConfig: lumberjack.Logger which was initialized at the beginning.
type EventLoggerEntry struct {
	EntryName        string                `yaml:"entryName" json:"entryName"`
	EntryType        string                `yaml:"entryType" json:"entryType"`
	EntryDescription string                `yaml:"entryDescription" json:"entryDescription"`
	EventFactory     *rkquery.EventFactory `yaml:"-" json:"-"`
	EventHelper      *rkquery.EventHelper  `yaml:"-" json:"-"`
	LoggerConfig     *zap.Config           `yaml:"zapConfig" json:"zapConfig"`
	LumberjackConfig *lumberjack.Logger    `yaml:"lumberjackConfig" json:"lumberjackConfig"`
}
Copy the code

EventLogger field

You can see that the EventLogger prints logs that contain fields, so let’s talk about those fields.

field details
endTime The end of time
startTime The start time
elapsedNano Event time overhead (Nanoseconds)
timezone The time zone
ids Contains eventId, requestId, and traceId. If the original data interceptor is started, or if event.setrequest () is called by the user, the new RequestId will be used and eventId will be exactly the same as RequestId. If the invocation chain interceptor is started, traceId will be logged.
app containsappName, appVersion, entryType entryName.
env Realm contains arch, AZ, Domain, hostname, localIP, OS, realm, region. Realm, region, az, domain fields. These fields come from system environment variables (REALM, REGION, AZ, DOMAIN). “*” means the environment variable is empty.
payloads Contains RPC-related information.
error Contains errors.
counters Do this via event.setCounter ().
pairs Operate via event.addpair ().
timing Operate via event.startTimer () and event.endTimer ().
remoteAddr RPC remote address.
operation RPC name.
resCode RPC return code.
eventStatus Ended 或者 InProgress

example

-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- endTime = 2021-11-27 T02:17: s15-39. 670807 + 08:00 StartTime = 2021-11-27 T02:17: s15-39. 670745 + 08:00 elapsedNano = 62536 timezone = CST ids={"eventId":"4bd9e16b-2b29-4773-8908-66c860bf6754"} app={"appName":"echo-demo","appVersion":"master-f948c90","entryName":"greeter","entryType":"EchoEntry"} Env = {" arch ", "amd64", "az" : "*", "domain" : "*", "hostname" : "lark. Local", "localIP" : "10.8.0.6", "OS" : "Darwin", "realm" : "*", "region ":" * "} Payloads = {" apiMethod ":" GET ", "apiPath" : "/ fairly rk/v1 / healthy", "apiProtocol" : "HTTP / 1.1", "apiQuery" : ""," userAgent ":" curl / 7.64.1}"  error={} counters={} pairs={} timing={} remoteAddr=localhost:61726 operation=/rk/v1/healthy resCode=200 eventStatus=Ended EOECopy the code

How to configure EventLoggerEntry in boot.yaml?

EventLoggerEntry will inject the Application name into the Event. The initiator extracts the Application name from the go.mod file. If there is no go.mod file, the initiator uses the default name.

Users can configure multiple EventLogger instances as required and access them by name.

Complete configuration:

---
eventLogger:
  - name: event-logger                 # Required
    description: "This is description" # Optional
    encoding: console                  # Optional, default: console, options: console and json
    outputPaths: ["stdout"]            # Optional
    lumberjack:                        # Optional
      filename: "rkapp-event.log"      # Optional, default: It uses <processname>-lumberjack.log in os.TempDir() if empty.
      maxsize: 1024                    # Optional, default: 1024 (MB)
      maxage: 7                        # Optional, default: 7 (days)
      maxbackups: 3                    # Optional, default: 3 (days)
      localtime: true                  # Optional, default: true
      compress: true                   # Optional, default: true
Copy the code

How do I get EventLogger in code?

Access by name.

// Access entry
rkentry.GlobalAppCtx.GetEventLoggerEntry("event-logger")

// Access event factory
rkentry.GlobalAppCtx.GetEventLoggerEntry("event-logger").GetEventFactory()

// Access event helper
rkentry.GlobalAppCtx.GetEventLoggerEntry("event-logger").GetEventHelper()

// Access lumberjack config
rkentry.GlobalAppCtx.GetEventLoggerEntry("event-logger").GetLumberjackConfig()
Copy the code

How do I use Events?

An Event is an interface that contains several methods. See: Event

Common methods:

// Get EventHelper to create Event instance
helper := rkentry.GlobalAppCtx.GetEventLoggerEntry("event-log").GetEventHelper()

// Start and finish event
event := helper.Start("demo-event")
helper.Finish(event)

// Add K/V
event.AddPair("key"."value")

// Start and end timer
event.StartTimer("my-timer")
event.EndTimer("my-timer")

// Set counter
event.SetCounter("my-counter".1)
Copy the code