Go-kit microservices: Service registration and discovery

In the microservice architecture, the original single service is split into multiple independent microservices, so the client cannot know the specific location of the service. Moreover, there are so many services and so many service addresses that operations personnel cannot work efficiently.

Therefore, a service registry was introduced in the microservices architecture to accept and maintain address information for individual services. The client or gateway can query the target service address through the registry, dynamically realize service access, and realize service load balancing.

For service registration and discovery, Go-Kit provides support for Consul, ZooKeeper, ETCD, and Eureka common registries by default.

An overview of the

Based on Consul, this paper will use the “client discovery mode” to conduct a practical exercise, which mainly includes the following points:

Consul Uses docker image (Progrium/Consul) in standalone deployment.
Arithmetic services registered to Consul with name Arithmetic;
Write discovery service Query service instance on Consul and load balancing based on RoundRibbon.

The idea of the example program in this paper is as follows: The arithmetic service is registered in Consul, while other parts remain unchanged; Discover the service to expose HTTP interface. After receiving the request (the received request content is stored in the Body and transferred in JSON mode), dynamically query the arithmetic service instance according to the mechanism of Go Kit, invoke the interface of the arithmetic service, and then return the response content. As shown below:

Start the consul

Modify thedocker/docker-compose.yml, as shown below (temporarily annotated with sections for Prometheus and Grafana).

version: '2'

services:
  consul:
    image: progrium/consul:latest
    ports:
      - 8400:8400
      - 8500:8500
      - 8600:53/udp
    hostname: consulserver
    command: -server -bootstrap -ui-dir /ui
Copy the code

Start the docker. Switch to the project directory on terminal and run the following command:

sudo docker-compose -f docker/docker-compose.yml up
Copy the code

Access through a browserhttp://localhost:8500If the following page is displayed, the startup is successful.

The service registry

Step-1: Code preparation

This example is adapted based on the arithmetic_monitor_demo code. First, copy the directory and rename it arithmetic_consul_demo; Create two directories named Register and Discover respectively. Move the original GO code file to the Register directory. The results are shown below:

In addition, you need to download the dependent third-party libraries UUID and Hashicorp/Consul

go get github.com/pborman/uuid
go get github.com/hashicorp/consul
Copy the code

Step-2: implement the registration method

Create register/register.go and add the register method to implement the registration logic to Consul. This method receives five parameters, namely the IP and port of registry Consul, the local IP and port of the arithmetic service, and the logging tool.

To create a registered object, use hashicorp/ Consul.

func NewRegistrar(client Client, r *stdconsul.AgentServiceRegistration, logger log.Logger) *Registrar
Copy the code

Therefore, there are three steps to implement Register: create consul client object; Create consul parameter configuration information for arithmetic service health check; Create the service configuration information that the arithmetic service registers with Consul. The code is as follows:

func Register(consulHost, consulPort, svcHost, svcPort string, Logger log.logger) (registar sd.logger) {// Connect to Consul client consulCfg := api.DefaultConfig()
		consulCfg.Address = consulHost + ":" + consulPort
		consulClient, err := api.NewClient(consulCfg)
		iferr ! = nil { logger.Log("create consul client error:". Err) os.exit (1)} client = consul.newClient (consulClient)} Check := api.agentServicecheck {HTTP:"http://" + svcHost + ":" + svcPort + "/health",
		Interval: "10s",
		Timeout:  "1s",
		Notes:    "Consul check service health status.",} port, _ : = strconv Atoi (svcPort) micro / / set the service to Consul registration information reg: = API. AgentServiceRegistration {ID:"arithmetic" + uuid.New(),
		Name:    "arithmetic",
		Address: svcHost,
		Port:    port,
		Tags:    []string{"arithmetic"."raysonxin"Registar = consul.newRegistrar (client, &reg, logger) registar = consul.newRegistrar (client, &reg, logger)return
}
Copy the code

Step-3: implements the health check interface

Step-2 shows that Consul regularly requests /heath of the arithmetic service to check the health status of the service, so we will add corresponding implementations from Service, endpoint, and transport.

At the interfaceServiceAdded interface methods inHealthCheck, and in turnArithmeticService,loggingMiddleware,metricMiddlewareAdd the implementation to.

// service Define a service interfacetypeService interface {// omit previous other methods // HealthCheck check Service health status HealthCheck() bool} // ArithmeticService implements HealthCheck // HealthCheck Implement Service Method // which is used to check the health status of the Servicetrue. func (s ArithmeticService) HealthCheck() bool {return true} // loggingMiddleware implements HealthCheck func(MW loggingMiddleware) HealthCheck() (result bool) {defer func(begin) time.Time) { mw.logger.Log("function"."HealthChcek"."result", result,
			"took", time.Since(begin),
		)
	}(time.Now())
	result = mw.Service.HealthCheck()
	return} // metricMiddleware implements HealthCheck func(MW metricMiddleware) HealthCheck() (result bool) {defer func(begin time.time) { lvs := []string{"method"."HealthCheck"} mw.requestCount.With(lvs...) .Add(1) mw.requestLatency.With(lvs...) .Observe(time.Since(begin).Seconds()) }(time.Now()) result = mw.Service.HealthCheck()return
}
Copy the code

inendpoints.goNew structure in:ArithmeticEndpoints. In the previous example, only one endpoint was used, so I used the structure directlyendpoint.Endpoint. The definition is as follows:

// ArithmeticEndpoint define endpoint
type ArithmeticEndpoints struct {
	ArithmeticEndpoint  endpoint.Endpoint
	HealthCheckEndpoint endpoint.Endpoint
}
Copy the code

Create request, response objects for the health check, and correspondingendpoint.EndpointEncapsulation method. The code is as follows:

// HealthRequest Health check request structuretypeHealthRequest struct{} // HealthResponse Health check response structuretype HealthResponse struct {
	Status bool `json:"status"'} // MakeHealthCheckEndpoint Create health check Endpoint func MakeHealthCheckEndpoint(SVC Service) endpoint.endpoint {return func(ctx context.Context, request interface{}) (response interface{}, err error) {
		status := svc.HealthCheck()
		return HealthResponse{status}, nil
	}
}
Copy the code

intransports.goAdded a health check interface to the/health.

// MakeHttpHandler make http handler use mux func MakeHttpHandler(ctx context.Context, endpoints ArithmeticEndpoints, Logger log.logger) http.handler {r := mux.newrouter () // void calculate/{type}/{a}/{b} code // Create health Check handler."GET").Path("/health").Handler(kithttp.NewServer( endpoints.HealthCheckEndpoint, decodeHealthCheckRequest, encodeArithmeticResponse, options... )),return r
}
Copy the code

Step-4: Modify main.go

Next, add health check and service registration related calling code to main.go to make the above change logic take effect.

Health check.

// Create health check Endpoint, Unadded flow limiting healthEndpoint := MakeHealthCheckEndpoint(SVC) // Encapsulate the arithmetic and health check Endpoint to ArithmeticEndpoints endpts := ArithmeticEndpoints{ ArithmeticEndpoint: endpoint, HealthCheckEndpoint: R := MakeHttpHandler(CTX, endpts, logger)Copy the code

Service registration. Prepare the environment variables required by the service, create a registered object, register with Consul before the service starts, and cancel the registration from Consul after the service exits. Only part of the code is posted below; the full code is available on Github.

Var (consulHost = flag.string)"consul.host".""."consul ip address")
	consulPort  = flag.String("consul.port".""."consul port")
	serviceHost = flag.String("service.host".""."service ip address")
	servicePort = flag.String("service.port".""."service port") ) // parse flag.Parse() // ... // Register(*consulHost, *consulPort, *serviceHost, *servicePort, logger) gofunc() {
	fmt.Println("Http Server start at port:"Registar.register () handler := r errChan < -http.listenAndServe ()":"+*servicePort, handler)
}()

go func() {
	c := make(chan os.Signal, 1)
	signal.Notify(c, syscall.SIGINT, syscall.SIGTERM)
	errChan <- fmt.Errorf("%s", <-c)}() error := < -errchan registar.deregister () fmt.println (error) registar.deregister () fmt.println (error)Copy the code

Step-5: Compile & run

Open terminal and switch to project directory. After executing go build./ Register, enter the following command to start the arithmetic service (register service) :

/ register-consul. host localhost -consul.port 8500-service. host 192.168.192.145 -service.port 9000Copy the code

If the following information is displayed, the arithmetic service has been successfully registered with Consul.

You can also view logs about consul call /health interface on the terminal where the registered service is running:

Service discovery

The work of Discover service is as follows: provide API service externally with REST interface/Calculate, and send JSON data to execute request by HTTP POST method. Query service instances registered in Consul on the endpoint. Then select the appropriate service instance to initiate request forwarding; After completing the request, request a response from the original client.

Kit/SD /Endpointer provides a service discovery mechanism defined and created as follows:

// Endpointer listens to a service discovery system and yields a set of
// identical endpoints on demand. An error indicates a problem with connectivity
// to the service discovery system, or within the system itself; an Endpointer
// may yield no endpoints without error.
typeEndpointer interface { Endpoints() ([]endpoint.Endpoint, error) } // NewEndpointer creates an Endpointer that subscribes to updates from Instancer src // and uses factory f to create Endpoints. If src notifies of an error, the Endpointer // keeps returning previously created Endpoints assuming they are still good, unless // this behavior is disabled via InvalidateOnError option. func NewEndpointer(src Instancer, f Factory, logger log.Logger, options ... EndpointerOption) *DefaultEndpointerCopy the code

We can see from the code comments that Endpointer finds system events by listening to the service and creates service endpoints (endpoints) on demand through the factory.

Therefore, we need to implement service discovery through Endpointer. In microservice mode, multiple instances of a service may exist. Therefore, you need to use the load balancing mechanism to select instances. In this case, the Kit, SD, and LB components in the Go-Kit toolkit are used (the components implement RoundRibbon and Retry function).

Step-1: Creates a Factory

Create the go file factory.go in the Discover directory to implement the logic of sd. factory, that is, convert the service instance to an endpoint, and invoke the target service in the endpoint. Here we encapsulate the arithmetic service directly, as follows:

func arithmeticFactory(_ context.Context, method, path string) sd.Factory {
	return func(instance string) (endpoint endpoint.Endpoint, closer io.Closer, err error) {
		if! strings.HasPrefix(instance,"http") {
			instance = "http://" + instance
		}

		tgt, err := url.Parse(instance)
		iferr ! = nil {return nil, nil, err
		}
		tgt.Path = path

		var (
			enc kithttp.EncodeRequestFunc
			dec kithttp.DecodeResponseFunc
		)
		enc, dec = encodeArithmeticRequest, decodeArithmeticReponse

		return kithttp.NewClient(method, tgt, enc, dec).Endpoint(), nil, nil
	}
}

func encodeArithmeticRequest(_ context.Context, req *http.Request, request interface{}) error {
	arithReq := request.(ArithmeticRequest)
	p := "/" + arithReq.RequestType + "/" + strconv.Itoa(arithReq.A) + "/" + strconv.Itoa(arithReq.B)
	req.URL.Path += p
	return nil
}

func decodeArithmeticReponse(_ context.Context, resp *http.Response) (interface{}, error) {
	var response ArithmeticResponse
	var s map[string]interface{}

	if respCode := resp.StatusCode; respCode >= 400 {
		iferr := json.NewDecoder(resp.Body).Decode(&s); err ! = nil {return nil, err
		}
		return nil, errors.New(s["error"].(string) + "\n")}iferr := json.NewDecoder(resp.Body).Decode(&response); err ! = nil {return nil, err
	}
	return response, nil
}
Copy the code

Step-2: creates an endpoint

Create the go file Discover /enpoints.go. According to the above analysis, the service discovery system is monitored on this endpoint, the instance is selected, and the executable endpoint.endpoint is finally returned. The implementation process is described below according to the code comments:

Func MakeDiscoverEndpoint(CTX context. context, client consul.Client, logger log.Logger) endpoint.Endpoint { serviceName :="arithmetic"
	tags := []string{"arithmetic"."raysonxin"}
	passingOnly := trueDuration := 500 * time.millisecond Instancer := consul.NewInstancer(client, Logger, serviceName, tags, PassingOnly) // Create sd.Factory Factory := arithmeticFactory(CTX,"POST"."calculate"Sd. factory endPointer := sd.NewEndpointer(instancer, factory, Logger) // Create a RoundRibbon balancer := lb.newroundrobin (endpointer) // Add retry for the load balancer, Endpoint retry := lb.retry (1, duration, balancer)return retry
}
Copy the code

Step-3: Create transport

Create the go file Discover/Transports.go. Endpoint, DecodeRequestFunc, and EncodeResponseFunc are required for discovery services using the REST/Calculate method via mux/Router. For convenience, I have copied the request and response structures and codec methods directly from the arithmetic service. The code looks like this:

func MakeHttpHandler(endpoint endpoint.Endpoint) http.Handler {
	r := mux.NewRouter()

	r.Methods("POST").Path("/calculate").Handler(kithttp.NewServer(
		endpoint,
		decodeDiscoverRequest,
		encodeDiscoverResponse,
	))

	returnR} // omit the entity structure and codec methodsCopy the code

Step-4: Write the main method

The next step is to string the above logic together in the main method and start the discovery service, listening on port 9001. Relatively simple, directly paste the code:

func mainVar (consulHost = flag.string () {consulHost = consulHost ();"consul.host".""."consul server ip address")
		consulPort = flag.String("consul.port".""."consul server port"Var logger log. logger {logger = log.newLogfmtLogger (os.stderr) logger = log.With(logger,"ts", log.DefaultTimestampUTC)
		logger = log.With(logger, "caller". Var client consulConfig := api.defaultconfig () consulconfig.address ="http://" + *consulHost + ":" + *consulPort
		consulClient, err := api.NewClient(consulConfig)

		iferr ! = nil { logger.Log("err". Err) os.exit (1)} client = consul.newClient (consulClient)} CTX := context.background () // Create an Endpoint MakeDiscoverEndpoint(CTX, Client, Logger) // Create transport layer r := MakeHttpHandler(discoverEndpoint) errc := make(chan Error) gofunc() {
		c := make(chan os.Signal)
		signal.Notify(c, syscall.SIGINT, syscall.SIGTERM)
		errc <- fmt.Errorf("%s"<-c)}() // Start listening on gofunc() {
		logger.Log("transport"."HTTP"."addr"."9001")
		errc <- http.ListenAndServe(": 9001", r)}() // Start running, wait for end logger.log ()"exit", <-errc)
}
Copy the code

Step-5: Compile & run

Switch to the Discover directory in the terminal, run Go Build to complete the build, and then start the discovery service using the following command (specifying the registry service address) :

./discover -consul.host localhost -consul.port 8500
Copy the code

Request to test

Use the postman request http://localhost:9001/calculate, setting the request information in the body, complete the test. As shown below:

conclusion

This article demonstrates go-Kit’s service registration and discovery capabilities using Consul as a registry. Since I do not have a thorough understanding of this part, I have been studying the design method of go-Kit discovery components in the process of code writing and this paper, and strive to explain clearly through codes and words. My level is limited, there is any mistake or improper place, please criticize and correct.

The code for an example of this article is arithmetic_consul_demo.

This article is first published in my wechat public account [Xi Yi Ang bar], welcome to scan code attention!

Go-kit microservices: Service registration and discovery

An overview of the

Start the consul

The service registry

Step-1: Code preparation

Step-2: implement the registration method

Step-3: implements the health check interface

Step-4: Modify main.go

Step-5: Compile & run

Service discovery

Step-1: Creates a Factory

Step-2: creates an endpoint

Step-3: Create transport

Step-4: Write the main method

Step-5: Compile & run

Request to test

conclusion

Related Posts

Prisma que

【 Design Mode Series 】 Builder mode

The dark secrets of BERT