Consul

Consul is an open source software from HashiCorp. Written in GO language, Consul provides the capabilities of service registration and discovery, configuration, and high availability solutions for multiple data centers. Distributed consistency is implemented using RAFT algorithm, and it is easy to integrate with Spring Cloud and other microservice frameworks. It is very simple to use, with simple, easy-to-use, pluggable and other characteristics. In short, Consul offers a complete service grid solution.

Consul has the following features and functions

• Service discovery: Clients can register with Consul for services, such as API services or mysql services, and other clients can use Consul to discover service providers. Consul supports registering and discovering services using DNS or HTTP.
• Runtime Health check: Consul client can provide any number of health check mechanisms that can be associated with a given service (” Yes Web server returns 200 OK “) or local node (” Memory utilization is less than 90% “). This information can be used to monitor the health of the cluster, and service discovery components can use this monitoring information to route traffic away from unhealthy services.
• KV storage: Applications can use Consul’s key/value store for any need, including dynamic configuration, feature marking, coordination, leader election, and more. It uses the HTTP API to make it easy to use.
• Secure service Communication: Consul can generate and distribute TLS certificates for services to establish mutual TLS connections.
• Multi-data Center: Consul supports multiple data centers. This means that Consul’s users don’t have to worry about building additional layers of abstraction to scale to multiple regions.

Consul principle

Each node that provides services runs Consul’s agent, which does not require the service to discover and obtain configured KV key pairs. The agent is only responsible for monitoring and checking. Agent nodes can communicate with one or more Consul Servers. Consul server is where data is stored and replicated. The server itself chooses the leader. Although Consul can run on one server, it is recommended to use 3 through 5 to avoid failure situations that result in data loss. You are advised to use a set of Consul servers for each data center.

If your component needs to discover services, you can query any Consul Server or any Consul client, and Consul client will automatically forward the query to Consul Server.

Infrastructure components that need to discover additional services or nodes can query any Consul server or any Consul agent. The proxy automatically forwards the query to the server. Each data center runs Consul server clusters. When a cross-data center service discovery or configuration request occurs, the local Consul server forwards the request to the remote data center and returns the result.

The term

• Agent Agent: is a daemon process running on each member in Consul cluster. Start by running Consul Agent. The Agent can run in client or Server mode. It is very simple to specify a node as a client or server, unless there are other Agent instances. All agents can run DNS or HTTP interfaces, check and synchronize services at run time.

• Client A Client is a proxy that forwards all RPCS to the server. This client is relatively stateless. The only background activity that client performs is joining the LAN Gossip pool. This has a minimal resource overhead and consumes only a small amount of network bandwidth.

• Server: A Server is an agent with an extended set of functions including participating in Raft elections, maintaining cluster state, responding to RPC queries, interacting with other data centers WAN Gossip and forwarding queries to the leader or remote data center.

• While the definition of a DataCenter is obvious, there are some fine details that must be considered. For example, in EC2, multiple available areas are considered to constitute a data center? We define the data center as a private, low latency and high bandwidth network environment. This does not include access to the public network, but for us, multiple available areas in the same EC2 can be considered part of a single data center.

• Consensus: In our documentation, we use Consensus to indicate agreement on the leader election and the order of transactions. Since these transactions are applied to finite state machines, Consensus implies duplicating the consistency of the state machine.

• Gossip: Consul builds on Serf, which provides a complete Gossip protocol for multicast purposes. Serfs provide membership, fault detection, and event broadcasting. More information is described in the Gossip document. This is enough to know that Gossip uses random point-to-point communication based on UDP.

• LAN Gossip contains all nodes in the same LAN or data center.

• WAN Gossip: This contains only servers. These servers are distributed in different data centers and usually communicate over the Internet or wan. RPC remote procedure call. This is a request/response mechanism that allows clients to request servers.

Let’s break down the diagram and describe each part. First, we can see that there are two data centers, labeled “1” and “2”. Consul has first-class support for multi-data centers and expects this to be a common occurrence.

In each data center, client and Server are mixed. Generally, 3-5 servers are recommended. This is based on the tradeoff between availability and performance in the event of a failure, as the more machines involved the slower consensus is reached. However, there is no limit to the number of clients and they can easily scale to thousands or tens of thousands.

All nodes in the same data center must join the Gossip protocol. This means that the Gossip protocol contains all the nodes of a given data center. This serves several purposes: First, there is no need to configure the server address on the client. Discovery is all automated. Second, the job of detecting node failures is not placed on the server, but distributed. This makes fault detection more scalable than heartbeat mechanisms. Third: It is used as a message layer to notify events, such as when the leader election occurs.

Servers in each data center are part of the Raft node collection. This means that they work together and elect a leader, a server with extra work. The leader is responsible for handling all queries and transactions. As part of the consistency protocol, transactions must also be replicated to all other nodes. Because of this requirement, when a non-leader server receives an RPC request, it forwards the request to the cluster leader.

The server node is also part of the WAN Gossip Pool. This Pool differs from a LAN Pool in that it is optimized for the higher latency of the Internet, and it only contains other Consul Server nodes. The purpose of this Pool is to allow data centers to discover each other in a low-touch manner. This makes it easy for a new data center to join the existing WAN Gossip. Because servers are running in this pool, it also supports cross-data center requests. When a server receives a request from another data center, it then forwards it to one of the servers in the correct data. The server is then forwarded to the local leader.

This leaves only a very low coupling between data centers, but requests across data centers are relatively fast and reliable due to fault detection, connection caching, and reuse.

Consul service registration discovery process

Consul is most widely used in the industry as a service registry. Like Eureka, Consul is a service registry. Its registration and discovery process is shown below:

On the above flowchart, there are three roles: service registry, service provider, and service consumer.

• When the service Provider is started, it sends a request to Consul to send its metadata information, such as host, IP address, application name, and health check, to Consul
• Consul receives the Provider’s registration and periodically sends health check requests to the Provider to check whether the Provider is healthy
• A service Consumer obtains the service registration list from Consul. When a service Consumer consumes a service, he or she obtains one or more service instances from Consul based on the application name to invoke the service.

Consul versus Eureka

As we know from the registry comparison table above, Consul and Eureka differ in that Consul guarantees CA while Eureka guarantees AP.

Consul strong consistency brings:

Service registration is a bit slower than Eureka. Consul’s RAFT protocol requires that more than half of the nodes on Consul have been successfully written to be considered registered. If the Leader fails, the entire Consul becomes unavailable during a re-election. Consistency is guaranteed at the expense of availability.

Eureka guarantees high availability (A) and ultimate consistency:

Registration of services is relatively fast, because there is no need to wait for the registration to go to other nodes, and there is no guarantee that the registration will succeed. [When data is inconsistent](juejin.cn/post/684490…) Although the registration information on A and B are not completely the same, each Eureka node can still provide services to the outside normally. In this case, if A fails to find the service information when requesting B, it can find the service information when requesting B. This ensures availability at the expense of consistency.

Consul download and install

Download address: www.consul.io/downloads.h…

Linux

Start command:

Nohup consul-server-bootstrap-expect 1-config-dir /etc/consul.d/ -data-dir/TMP/consul-ui-bind =0.0.0.0 >> nohup agent-server-bootstrap-expect 1-config-dir /etc/consul.d/ -data-dir/TMP/consul-ui-bind =0.0.0.0 >> /var/opt/consul/consul.log 2>&1Copy the code

Mac

#The installation
brew install consul
#Modify Consul startup parameters
vim /usr/local/opt/consul/homebrew.mxcl.consul.plist
#Modify the ProgramArguments section<key>ProgramArguments</key><array><string>/usr/local/opt/consul/bin/consul</string><string>agent</string><string>-server < / string > < string > - the bootstrap < / string > < string > - advertise < / string > < string > 127.0.0.1 < / string > < string > - data - dir < / string > < stri ng>./data</string><string>-ui</string></array>#Consul started
bashbrew services start consul
Copy the code

Accessing the Web UI

Visit: http://localhost:8500

- Bootstrap: indicates the startup mode. In this mode, a node can elect itself as the leader. A data center can have only one node in this startup mode. After a cluster is started, you are not advised to use this mode for newly started nodes. -bootstrap-expect: specifies the number of service nodes required by a data center. This parameter is optional and must match the actual number of service nodes. Consul waits until the service nodes in the data center meet the Settings before starting the cluster service. Initialize the leader election. Cannot be used with bootstrap. It must work with the -server configuration. -bind: specifies the internal communication address of the bind. The default value is 0.0.0.0. That is, all local addresses propagate the first available IP address into the cluster. [::]ipv6, TCP UDP, same port. Firewall configuration. -client: client mode, HTTP DNS, default 127.0.0.1, loopback token URL -config-file: configuration file location -config-dir: The configuration file folder will load all configuration files under it,.json or.hcl files, in alphabetical order. Multiple configurations of this configuration can be configured to load multiple folders, and configurations of subfolders will not be loaded. -config-format: indicates the format of the configuration file. If the format is configured, the corresponding file is loaded. If this parameter is not configured, Consul automatically identifies data. -data-dir: indicates the status data storage folder, which is required by all nodes. The folder location does not receive consul consul when the node is restarted. You must be able to use the OS file lock. In the UNIX-based operating system, the folder file permission is 0600. Data center name, default dC1, all nodes in a data center must be in the same LAN. -dev: the development mode is memory server mode, with all persistent options removed. -disable-host-node-id: the node ID is not generated using the host information. This parameter is applicable when multiple instances are deployed on the same server for testing. 8600-enable-script-checks: specifies whether scripts are allowed to perform health checks. The default value is false. The best option is to configure enable acl-encrypt: encrypt Consul network communication encryption key, base64 encryption, 16 bits; Consul KeyGen is generated. Each instance in the cluster must provide the same, only once, and then be saved to a data file. Restart automatic loading. Consul is provided after the consul node is started and will be ignored. - HCL: Adds the HCL configuration and merges it with the existing configuration. You can use multiple of this configuration. -http-port: indicates the HTTP API port. The default value is 8500. Applicable to cloud environment. -log-file: indicates the log record file. If the file name is not specified, the Consul- time timestamp is default. Consul monitor monitor-log-depressury-bytes: specifies the size threshold for generating new log files. - log-rotation-rotation: new log generation time threshold -join: IP address of other nodes to be added. You can add multiple nodes. -retry-join: retries are performed when the added node is considered normal. Ipv4, ipv6, dns-retry-interval: specifies the retry interval. The default value is 30s-retry-max. -join-wan, -retry-join-wan, -retry-interval-wan, -retry-max-wan-node: specifies the node name. The default host name is -node-id. Node ID, -pid-file: consul Specifies the file location of the PID in consul storage, which is used to actively send signals. Such as stop node, overload configuration, etc. -protocol: indicates the protocol used during the upgrade. Consul -v Checking the protocol version -raft-protocol: raft protocol version is used. Default: 3-raft -snapshot-threshold: threshold for the number of snapshot submission times when raft performs the snapshot Generally, this parameter is not required. For IO intensive applications, it can be set to a higher value. Avoid snapshots for all nodes at the same time. If the value is too large, the corresponding log file becomes larger, and node restart takes a longer time. The default value is 16384 after 1.1.0 and 8192 before. -raft-snapshot-interval: specifies the snapshot interval, which affects the same configuration as the previous one. The default value is 30s after 1.1.0 and 5s before 1.1.0. -rejoin: the node will try to join the cluster again. -server: indicates the server node mode. -server-port: indicates the RPC port of the server, provided after V1.2.2. -non-voting-server: the service node does not participate in elections and accepts log replication for horizontal expansion and service query requests. (Similar to the ZooKeeper observer node) - Syslog: in Linux OSX, logs are output to system logs. - UI: indicates the built-in Web UI. -ui-dir: indicates the web UI resource folder. If this configuration is used, the -ui configuration is not required and cannot be used.Copy the code

The specific parameters: www.consul.io/docs/agent/…

Default port:

• Server RPC: 8300, TCP by default.
• Serl LAN: Handles LAN gossip, default 8301, TCP UDP.
• Serl WAN: Handles LAN gossip, default 8302, TCP UDP.
• HTTP API: 8500, TCP.
• DNS: 8600, TCP,UDP.

The Consul client

Add the dependent

Add POM dependencies

<! -- Monitor, provide health check -->
<dependency>
	<groupId>org.springframework.boot</groupId>
	<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
	<groupId>org.springframework.cloud</groupId>
	<artifactId>spring-cloud-starter-consul-discovery</artifactId>
</dependency>
Copy the code

Modify the configuration

bootstrap.yml

spring:
 cloud:
  consul:
   host: xxx.xxx.xxx.xxx # address
   port: 8500 # port
   discovery:
   	register: true # Whether to register with Consul
    prefer-ip-address: true IP priority, register with CONSUL in IP mode
    instance-id: ${spring.application.name}:${spring.cloud.client.ip-address} # instance name
    healthCheckInterval: 15s  # Health check interval
    health-check-url: http://${spring.cloud.client.ip-address}:${server.port}/actuator/  Health check address
Copy the code

Start the class

Add the @EnableDiscoveryClient annotation, or just @SpringCloudApplication

@SpringCloudApplication
public class GatewayApplication {
    public static void main(String[] args) { SpringApplication.run(GatewayApplication.class, args); }}Copy the code

Enable configuration items related to the configuration file

spring:
  application:
    name: consul-provider
  cloud:
    consul:
      config:
          enabled: true
          format: yaml           
          prefix: config     
          profile-separator: ':'    
          data-key: data 
Copy the code

Web site visit consul KV storage management interface, i.e. http://localhost:8500/ui/dc1/kv, create a configuration record.

Dynamic refresh configuration is implemented using @refreshScope

@RestController
@RefreshScope
public class FooBarController {

    @Value("${foo.bar}")
    String fooBar;

    @GetMapping("/foo")
    public String getFooBar(a) {
        returnfooBar; }}Copy the code

Parameters that

spring:
 cloud:
  consul:
   host: xxx.xxx.xxx.xxx # address
   port: 8500 # port
   discovery:
   	register: true # Whether to register with Consul
   	serviceName: service-producer
    prefer-ip-address: true IP priority, register with CONSUL in IP mode
    tags: Version = 1.0  # key-value
    instance-id: ${spring.application.name}:${spring.cloud.client.ip-address} # instance name
    healthCheckInterval: 15s  # Health check interval
    health-check-url: http://${spring.cloud.client.ip-address}:${server.port}/actuator/  Health check address
Copy the code

Registration service name attribute for spring. Cloud. Consul. Discovery. The serviceName, and is no longer like Eureka used in spring. The application. The name = xx.

API

• http://localhost:8500/v1/agent/checks: for all current health service

• http://localhost:8500/v1/acl/bootstrap: create a token

• http://localhost:8500/v1/acl/create: create an ACL configuration

• http://127.0.0.1:8500/v1/agent/service/deregister/my-service-id: offline services

See also: API details

ACL

Consul Server configuration

To generate the token

Visit http://localhost:8500/v1/acl/bootstrap in the PUT way, take SecretID or ID you will find the two values are the same.

{
    "ID": "e292b56b-eee4-76c6-07a1-af7a81fc8c38"."AccessorID": "3aea8f1b-f30a-5a0a-3970-14135c4ce1a3"."SecretID": "e292b56b-eee4-76c6-07a1-af7a81fc8c38"."Description": "Bootstrap Token (Global Management)"."Policies": [{"ID": "00000000-0000-0000-0000-000000000001"."Name": "global-management"}]."Local": false."CreateTime": "The 2019-12-16 T16:11:54. 159198 + 08:00"."Hash": "oyrov6+GFLjo/KZAfqgxF/X4J/3LX0435DOBy9V22I0="."CreateIndex": 1061."ModifyIndex": 1061
}
Copy the code

Example Modify the Consul server configuration file

In the directory where Consul is located (/etc/consul.d), create an acl.json file and perform the following operations to restart Consul.

{
  "acl_master_token": "e292b56b-eee4-76c6-07a1-af7a81fc8c38"."acl_default_policy": "deny"."server": true."log_level": "INFO"."bootstrap_expect": 1."client_addr": "0.0.0.0"
}
Copy the code

You can see the ACL_token configuration item online, which is used for Consul client authentication.

Do not name the acl.json file arbitrarily, because configuration items may be confused

Creating AN ACL Rule

Visit http://localhost:8500/v1/acl/create to PUT way

The request parameters are as follows:

{
  "Name": "AgentToken"."Type": "client"."Rules": "node \"\" { policy = \"read\" } node \"\" { policy = \"write\" } service \"\" { policy = \"read\" } service \"\" { policy = \"write\" } key \"\" { policy = \"read\" } key \"\" { policy = \"write\" } agent \"\" { policy = \"read\" } agent \"\" { policy = \"write\" } session \"\" { policy = \"read\" } session \"\" { policy = \"write\" }"
}
Copy the code

Check whether the authentication takes effect

#On the command line, return null
consul members
#Return list data
consul members -token e292b56b-eee4-76c6-07a1-af7a81fc8c38
Copy the code

Consul Client configuration

Spring Cloud Consul client configuration

Configure the acL-token configuration item on Consul client. You are advised to save the configuration item in the bootstrap.yml file

spring:
  cloud:
    consul:
      discovery:
        enabled: true
        healthCheckPath: ${management.server.servlet.context-path}/health
        healthCheckInterval: 15s
        prefer-ip-address: true
        register: true
        acl-token: e292b56b-eee4-76c6-07a1-af7a81fc8c38
      host: localhost
      port: 8500
Copy the code

Rules

Rule configuration description in step 3 above.

Policy	Scope
agent	Utility operations in the Agent API, other than service and check registration
event	Listing and firing events in the Event API
key	Key/value store operations in the KV Store API
keyring	Keyring operations in the Keyring API
node	Node-level catalog operations in the Catalog API.Health API.Prepared Query API.Network Coordinate API, and Agent API
operator	Cluster-level operations in the Operator API, other than the Keyring API
query	Prepared query operations in the Prepared Query API
service	Service-level catalog operations in the Catalog API.Health API.Prepared Query API, and Agent API
session	Session operations in the Session API

If the Token is invalid, you can view Consul Access Control (ACLS) parameters.

See: Official ACL Configuration

Consul Web UI security

Consul itself does not provide a security guarantee for consul web UI. As long as the firewall allows, anyone on the exnet can access Consul web UI, which is dangerous. Auth_basic is used to ensure consul web UI security.

1. A server running Consul Agent in server mode is configured with a network policy to allow only other nodes within the Intranet to access port 8500.

2. If a node running Consul Agent in client mode has web UI enabled, only address 127.0.0.1 is bound to the node. It can connect to Consul Server Agent using port 8500, but must use a client token or a Management token to use Consul functions.

3. On the Intranet, use Nginx or Apache as a reverse proxy to access port 8500 on the Consul Server Agent node and configure auth_basic authentication on nginx or Apache. For the configuration of reverse proxy and Auth_BASIC authentication, see the following:

   yum install -y httpd-tools
   htpasswd -c /etc/nginx/htpsswd consul_access You will be asked to enter your password
   # Configure nginxUpstream {server 10.12.142.216:8500; Server 10.12.142.217:8500; Server 10.12.142.218:8500; } server { listen 18500; server_name consul.xxxx.com; location / { proxy_pass http://consul; proxy_read_timeout 300; proxy_connect_timeout 300; proxy_redirect off; auth_basic"Restricted"; auth_basic_user_file /etc/nginx/htpasswd; }}Copy the code

4. Configure network policies. Only the reverse proxy address of Nginx is allowed to be accessed from the Extranet. Auth_basic authentication information is required for access. In addition, the client token must be used when consul function is used.

Spring Cloud Consul configuration

Core parameter

Configuration items	The default value
spring.cloud.consul.enabled	true
spring.cloud.consul.host	localhost
spring.cloud.consul.port	8500

For other parameters, see Spring Cloud Consul from Getting Started to Mastering

The problem

1. There are Critical problems

The cause may be that the spring-boot-starter-actuator dependencies are not referenced

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
Copy the code

The Spring Cloud Consul error message is as follows. The SC application is not configured with acL-token. Bootstrap and Application are not related. Or debug ConsulDiscoveryProperties class the configuration initialization.

com.ecwid.consul.v1.OperationException: OperationException(statusCode=403, statusMessage='Forbidden', statusContent='Permission denied')
Copy the code

Develop reading

Greenwich Overview of Spring Cloud Consul

Consul Getting Started

Spring Cloud Consul

Springcloud microservice system (ii) – 2.0G Consul registry establishment

Consul configuration/KV/ACL

Spring Cloud Consul goes from getting started to mastering

Consul Basics

Consul API

Consul Access Control (ACLS)