reference

  • The official documentation

Introduction to Consul: Consul

What is Consul?

  • Consul is a service mesh solution providing a full featured control plane with service discovery, configuration, and segmentation functionality.

The key features of Consul are Core features/functions

  • Service Discovery Service Discovery: Clients of Consul can register a service, such as api or mysql, and other clients can use Consul to discover providers of a given service. Using either DNS or HTTP, applications can easily find the services they depend upon.
  • Health Checking Service Health Checking: Consul clients can provide any number of health checks, either associated with a given service (“is the webserver returning 200 OK”), or with the local node (“is memory utilization below 90%”). This information can be used by an operator to monitor cluster health, and it is used by the service discovery components to route traffic away from unhealthy hosts.
  • Multi Datacenter supports multiple DATA centers: Consul supports multiple datacenters out of the box. This means users of Consul do not have to worry about building additional layers of abstraction to grow to multiple regions.

Basic Architecture of Consul

  • Consul Agent Node running the Consul process: Every node that provides services to Consul runs a Consul agent. Running an agent is not required for discovering other services or getting/setting key/value data. The agent is responsible for health checking the services on the node as well as the node itself.
  • Consul Server provides: The Consul servers are where data is stored and replicated. The servers themselves elect a leader. While Consul can function with one server, 3 to 5 is recommended to avoid failure scenarios leading to data loss. A cluster of Consul servers is recommended for each datacenter.
  • catalog: The catalog maintains the high-level view of the cluster, including which services are available, which nodes run those services, health information, and more. The catalog is used to expose this information via the various interfaces Consul provides, including DNS and HTTP.

Consul Glossary terms

  • Agent: Runs the Consul master process and is the basic unit that makes up the Consul cluster. It has two roles: client and Server.

An agent is the long running daemon on every member of the Consul cluster. It is started by running consul agent. The agent is able to run in either client or server mode. Since all nodes must be running an agent, it is simpler to refer to the node as being either a client or server, but there are other instances of the agent. All agents can run the DNS or HTTP interfaces, and are responsible for running checks and keeping services in sync.

  • Client/Server: two types of nodes that make up consul cluster. In addition, the server has two roles: leader and follower.

Client is responsible for forwarding RPC requests to ‘server’ and participating in the INTERNAL LAN gossip data synchronization in DC: A client is an agent that forwards all RPCs to a server. The client is relatively stateless. The only background activity a client performs is taking part in the LAN gossip pool. This has a minimal resource overhead and consumes only a small amount of network bandwidth.

Server is responsible for RPC request response, cluster status management, and participating in WAN Gossip data synchronization between DCS: A server is an agent with an expanded set of responsibilities including participating in the Raft quorum, maintaining cluster state, responding to RPC queries, exchanging WAN gossip with other datacenters, and forwarding queries to leaders or remote datacenters.

  • Datacenter Datacenter

We define a datacenter to be a networking environment that is private, low latency, and high bandwidth. This excludes communication that would traverse the public internet, but for our purposes multiple availability zones within a single EC2 region would be considered part of a single datacenter.

  • Consensus consistency: Consul uses the Raft consistency protocol, which involves leader elections and sequential definition of transactions.

When used in our documentation we use consensus to mean agreement upon the elected leader as well as agreement on the ordering of transactions. Since these transactions are applied to a finite-state machine, our definition of consensus implies the consistency of a replicated state machine. Consensus is described in more detail on Wikipedia, and our implementation is described here.

  • Gossip: Data synchronization protocol used by Consul, including the LAN Gossip inside DC and the WAN Gossip across DC.

Consul is built on top of Serf which provides a full gossip protocol that is used for multiple purposes. Serf provides membership, failure detection, and event broadcast. Our use of these is described more in the gossip documentation. It is enough to know that gossip involves random node-to-node communication, primarily over UDP.

  • LAN gossip: Refers to the LAN gossip pool which contains nodes that are all located on the same local area network or datacenter.
  • WAN gossip: Refers to the WAN gossip pool which contains only servers. These servers are primarily located in different datacenters and typically communicate over the internet or wide area network.
  • RPC: Calls between the client and server

Remote Procedure Call. This is a request / response mechanism allowing a client to make a request of a server.

Required Ports Default port

  • Consul uses UDP and TCP at the transport layer and requires at least six default ports to function properly

Consul requires up to 6 different ports to work properly, some on TCP, UDP, or both protocols. Below we document the requirements for each port.

reference

Consul vs. Other Software Consul vs. Other Cloud native Components (TODO)

  • Consul vs. ZooKeeper
  • Consul vs. Serf
  • Consul vs. Eureka
  • Consul vs. Istio