Author | Song Ruiguo (Dust drunk) source | Erda public account

The Erda Infra microservices framework evolved from the Erda project and is fully open source. Erda builds large and complex projects based on the Erda Infra framework. This article will comprehensively and deeply analyze the architecture design of Erda Infra framework and how to use it.

  • Erda Infra: github.com/erda-projec…
  • Erda: github.com/erda-projec…

background

With the rapid development of Internet technology, many large-scale systems have gradually evolved from single applications to micro-service systems.

Single application

The advantage of a single application is that it’s fast to develop and easy to deploy. We can quickly build an application and get it online without much thought. However, with the development of the business, the single program gradually becomes complicated and chaotic. It is very easy to change bugs, and the volume is getting bigger and bigger. When the business volume comes up, it is easy to crash.

Microservice Architecture

Large systems often adopt a microservice architecture, which divides complex systems into multiple services with loose coupling and high internal cohesion. At the same time, microservices architecture presents some challenges. The increasing number of services poses a challenge to the stability of the entire system. For example, how to deal with the failure of a service, how to communicate between services, and how to observe the overall system status. As a result, a variety of microservices frameworks were created, using various technologies to address the problems of microservices architecture. Spring Cloud is a comprehensive framework for microservices architecture in the Java domain.

Cloud platform

Spring Cloud offers a number of technical solutions, but it is still expensive for enterprises to operate and maintain. Enterprises need to maintain a variety of middleware and many micro-services, so there are a variety of cloud services, cloud platforms. Erda (github.com/erda-projec…). It is a one-stop PaaS platform to manage the whole life cycle of enterprise software system in the development stage and operation and maintenance stage, and can solve various problems brought by microservices in each stage. Erda itself is also a very large system, which is designed with microservice architecture. It also faces the problems brought by microservice architecture and puts forward more demands on the system. We hope to achieve:

  • The system is highly modular
  • The system has high scalability
  • Suitable for multi-player development mode
  • At the same time, support HTTP, gRPC interface, can automatically generate API Client

On the other hand, the development language of Erda is Golang, which is a mainstream development language in the cloud native field, especially suitable for the development of basic components. Many projects such as Docker, Kubernetes, Etcd and Prometheus also use Golang for development. Unlike Spring Cloud in Java, there is no dominant microservices framework in Golang’s ecosystem. We can find many Web frameworks, GRPC frameworks, etc., which provide a lot of tools, but don’t tell you how to design a system. It doesn’t help you decouple the modules in the system. Against this background, we developed the Erda Infra framework.

Erda Infra Microservices framework

A large system is generally composed of multiple applications, and an application contains multiple modules. The general application structure is shown in the figure below:

There are some problems with this structure:

  • Code coupling: One of the most coupled places is at the very beginning of the program, where all the configuration is read, all the modules are initialized, and some asynchronous tasks are started.
  • Dependency passing: Because of the dependencies between modules, it must be initialized in a certain order, including database Client, etc., and must be passed through the layers.
  • Poor scalability: adding or deleting a module is not so convenient, and it is easy to affect other modules.
  • Not good for multiplayer: If modules in an application are developed by more than one person, it is easy to influence each other. Debugging one module requires starting all modules in the entire application.

We’ll address these issues in a few steps.

Build a modul driven application

We can divide the whole system into small function points, and each small function point corresponds to a micro-module. The whole system is like a puzzle, building blocks, free combination of various functional modules into a large module as an independent application.

This also means that we don’t have to worry about having too many services scattered across the system, and we just need to focus on breaking down the functionality itself. Microservices exist not only between processes across nodes, but also within a process.

We use the Erda Infra framework to define a module:

package example

import (
	"context"
	"fmt"
	"time"

	"github.com/erda-project/erda-infra/base/logs"
	"github.com/erda-project/erda-infra/base/servicehub"
)

// Interface Provides the functions of the module in the form of an Interface
type Interface interface {
	Hello(name string) string
}

// config Specifies the declarative configuration definition
type config struct {
	Message string `file:"message" flag:"msg" default:"hi" desc:"message to print"`
}

// Provider represents a module
type provider struct {
	Cfg *config     // The framework will be injected automatically
	Log logs.Logger // The framework will be injected automatically
}

// Init initializes the module. Optional, if it exists, it will be called automatically by the framework
func (p *provider) Init(ctx servicehub.Context) error {
	p.Log.Info("message: ", p.Cfg.Message)
	return nil
}

// Run Starts the asynchronous task. Optional, if it exists, it will be called automatically by the framework
func (p *provider) Run(ctx context.Context) error {
	tick := time.NewTicker(3 * time.Second)
	defer tick.Stop()
	for {
		select {
		case <-tick.C:
			p.Log.Info("do something...")
		case <-ctx.Done():
			return nil}}}// Hello implements the interface
func (p *provider) Hello(name string) string {
	return fmt.Sprintf("hello %s", p.Cfg.Message)
}

func init(a) {
	// Register the module
	servicehub.Register("helloworld", &servicehub.Spec{
		Services:    []string{"helloworld-service"}, // Represents the list of services for the module
		Description: "here is description of helloworld",
		ConfigFunc:  func(a) interface{} { return &config{} }, // The configured constructor
		Creator: func(a) servicehub.Provider { // The constructor of the module
			return &provider{}
		},
	})
}

Copy the code

When we have defined many of these modules, we can start the module with a main function:

package main

import(_"... /example" // your package import path
	"github.com/erda-project/erda-infra/base/servicehub"
)

func main() {
	servicehub.Run(&servicehub.RunOptions{
		ConfigFile: "example.yaml"})},package main

import (
	"github.com/erda-project/erda-infra/base/servicehub"
	_ "... /example" // your package import path
)

func main() {
	servicehub.Run(&servicehub.RunOptions{
		ConfigFile: "example.yaml"})},Copy the code

Then, we use a configuration file example.yaml to determine which modules we start:

# example.yaml
helloworld:
	message: "erda"
Copy the code

​

Tip: There are also built-in configuration options, see ServiceHub. RunOptions for definitions.

The advantages of this approach are as follows:

  • Micro-modular programming only needs to care about its own function, which makes it easier to achieve high cohesion and low coupling.
  • Declarative configuration definitions, without caring about the configuration read steps, the framework implements configuration read in a variety of ways.
  • You don’t need to care about how other modules are initialized, and you don’t need to care about the initialization sequence of the entire application. You just need to focus on your own initialization steps.
  • Asynchronous task management, the framework will handle process signals, gracefully shut down module tasks.
  • The system height can be configured, any module can be configured independently, and a module can be started separately for debugging.

Dependencies between modules

As one of the problems faced by micro services, there are complex calls between services, and there are objective dependencies. After modularizing functions, how do we resolve dependencies between modules? Erda Infra gives us the way to do dependency injection. Before introducing dependency injection, let’s look at some concepts:

  • Service represents a function that can be used by other modules or other systems.
  • Provider: represents the Provider of services. It provides zero or more services. It is equivalent to a module or a set of services.
  • A Provider can depend on zero or more services.

We can define the dependent Service type as a field on the Provider, and the framework will automatically inject the dependent Service instance. For example, we define a module 2 that references the HelloWorld-service provided by the HelloWord module defined in the previous section:

package example2

// The following ellipses are not critical code

import (
	"... /example" // your package import path
	"github.com/erda-project/erda-infra/base/servicehub"
)

type provider struct {
	Example example.Interface `autowired:"helloworld-service"` // The framework will automatically inject instances
}

func (p *provider) Init(ctx servicehub.Context) error {
	p.Example.Hello("i am module 2")
	return nil
}

func init(a) {
	// Register module...
}

Copy the code

To think about it, why not rely on providers directly, but rely on services? Because the same Service can be provided by multiple providers with different implementations. Just as we rely on an interface rather than an implementation class, we can define the Service interface types we depend on in a common place, which are interpended by different Provider implementations. The caller does not need to care about the implementation, and we can switch between different implementations through configuration. This enables decoupling between modules.

The framework can do dependency injection by the name of the Service declared by AutoWired, or by the interface type, as shown in the examples in the Erda Infra repository. The framework analyzes the dependencies between modules to determine the initialization order of each module, so when we write a module, we don’t need to care about the initialization order of all modules in the program.

Build HTTP + gRPC services across processes

Once the dependencies between modules have been resolved, let’s consider how to communicate across processes.

We often need to migrate some functional modules to another application or split a large application into several smaller applications due to some architectural adjustments or other reasons. On the other hand, to realize all the small functions of the whole system arbitrarily combined into a large module, must also involve cross-process communication. The good news is that we can do interface-oriented programming, where modules are dependent on each other through the Service interface, so the interface can be either a local module or a remote module. Erda Infra can decouple modules and solve the problem of cross-process communication. By defining the ProtoBuf API, the framework provides modules the ability to support both HTTP and gRPC interfaces:

The framework also provides cli tools to help us generate related code:

Let’s look at an example. First, create a greeter. Proto file and define a GreeterService:

syntax = "proto3";

package erda.infra.example;
import "google/api/annotations.proto";
option go_package = "github.com/erda-project/erda-infra/examples/service/protocol/pb";

// the greeting service definition.
service GreeterService {
  // say hello
  rpc SayHello (HelloRequest) returns (HelloResponse)  {
    option (google.api.http) = {
      get: "/api/greeter/{name}"}; }}message HelloRequest {
  string name = 1;
}

message HelloResponse {
  bool success = 1;
  string data = 2;
}
Copy the code

​

The second step is to compile the protocol code and Client module through the GoHub tool provided by Erda Infra.

cd protocol
gohub protoc protocol *.proto 
Copy the code

Protocol/PB is the protocol code, and protocol/client is the third step of the client code. With the protocol code, it is necessary to implement the corresponding service interface, and the code template of the interface is generated through goHub:

cdserver/helloworld gohub protoc imp .. /.. /protocol/*.protoCopy the code

The contents of the greeter.service.go file are as follows:

package example

import (
	"context"

	"github.com/erda-project/erda-infra/examples/service/protocol/pb"
)

type greeterService struct {
	p *provider
}

func (s *greeterService) SayHello(ctx context.Context, req *pb.HelloRequest) (*pb.HelloResponse, error) {
	// TODO:Writing business logic
	return &pb.HelloResponse{
		Success: true,
		Data:    "hello " + req.Name,
	}, nil
}

Copy the code

With that, we can start writing our business logic in the module. Once we have written the implementation of the interface, we have both the HTTP and gRPC interfaces, which of course can be selectively exposed.

So how does a newly written module get referenced by other modules? Here’s an example:

package caller

import (
	"context"
	"time"

	"github.com/erda-project/erda-infra/base/logs"
	"github.com/erda-project/erda-infra/base/servicehub"
	"github.com/erda-project/erda-infra/examples/service/protocol/pb"
)

type config struct {
	Name string `file:"name" default:"recallsong"`
}

type provider struct {
	Cfg     *config
	Log     logs.Logger
	Greeter pb.GreeterServiceServer // Provided by a local or remote module
}

// Example of calling the GreeterService service
func (p *provider) Run(ctx context.Context) error {
	tick := time.NewTicker(3 * time.Second)
	defer tick.Stop()
	for {
		select {
		case <-tick.C:
			resp, err := p.Greeter.SayHello(context.Background(), &pb.HelloRequest{
				Name: p.Cfg.Name,
			})
			iferr ! =nil {
				p.Log.Error(err)
			}
			p.Log.Info(resp)
		case <-ctx.Done():
			return nil}}}func init(a) {
	servicehub.Register("caller", &servicehub.Spec{
		Services:     []string{},
		Description:  "this is caller example",
		Dependencies: []string{"erda.infra.example.GreeterService"},
		ConfigFunc: func(a) interface{} {
			return &config{}
		},
		Creator: func(a) servicehub.Provider {
			return &provider{}
		},
	})
}

Copy the code

Where pb.GreeterServiceServer is an interface generated by the ProtoBuf file, the caller does not need to care whether the implementation of the interface is provided by the local module or the remote module, this can be determined by the configuration file. When it is implemented by a local module, the local implementation function is called through the interface; When it is provided by a remote module, it is called through gRPC.

Example complete code: github.com/erda-projec… .

Module generalization

Erda Infra provides a number of off-the-shelf generic modules, right out of the box.

The httpServer module provides an effect similar to that of the Spring MVC Controller, which can write handlers with arbitrary arguments instead of the fixed HTTP.HandlerFunc form. Every program probably needs interfaces such as Health, Pprof, and so on, and we can have them just by importing the corresponding modules. In the same way, developers can develop more common business modules distributed in different warehouses for use by other business systems, which can greatly improve the reusability of functional modules.

conclusion

Erda Infra is a framework that can quickly build modem-driven systems that address many of the issues posed by microservices. In the future, there will be more generic modules to solve problems in different scenarios, which will improve development efficiency to a greater extent. If you want to know more about Erda, please add the little assistant wechat (Erda202106) to the communication group discussion, or click the link in the bottom to learn more!

  • Erda Github: github.com/erda-projec…
  • Erda Cloud website: www.erda.cloud/