There are some common architectures in distributed systems that can help describe how core services work. Spring Cloud provides powerful tools to enhance Spring Boot applications to implement these architectures.

Config Service Spring Cloud Config is a horizontally scalable centralized configuration service for distributed systems. It uses the pluggable repository layer that currently supports local storage, Git, and Subversion.

In this project, native profile is used, which loads configuration files from the local classpath. You can view shared’s directory in the Config service resource. Now, when the notification service requests its configuration, the configuration service responds with shared/notification-service.yml and shared/application.yml (shared among all client applications).

Clients simply build spring Boot applications with spring-cloud-starter-config dependencies, and automatic configuration will do the rest.

Now, you don’t need to use any embedded properties in your application. Simply provide the bootstrap.yml application name and configuration service URL:

Spring: Application: name: notification-service cloud: config: uri: http://config: 8888 fail-fast: trueCopy the code

With Spring Cloud Config, you can change the application configuration on the fly. For example, the EmailService bean is annotated with @refreshScope. This means that you can change the email text and subject without rebuilding and restarting the notification service application.

First, change the required properties in the Config server. Then, to refresh Notification service request: curl -h “Authorization: Bearer # token#” – XPOST http://127.0.0.1:8000/notifications/refresh

In addition, you can automate this process using Repository webhooks

Summary: * *

• But dynamic refreshing has some limitations.@RefreshScopeNot with@ConfigurationClass is used together and does not affect@Scheduledmethods
• fail-fastProperty means that the Spring Boot application will fail immediately if it cannot connect to Config
  
  The Service. This is very useful when launching all applications
• There are significant safety considerations as follows

The Auth Service fully abstracts responsibility for authorization to a separate server, which grants the OAuth2 token to the back-end resource service. The Auth server is used for user authorization and secure machine-to-machine communication within the perimeter.

In this project, I use user authorization of the Password Credentials authorization type (because it is only used by the local PiggyMetrics UI) and Client Credentials to grant microservice authorization.

Spring cloud security provides convenient annotation and automatic configuration, making it easy to implement from both the server and client side. You can learn more in the documentation and check the configuration details in the Auth Server code.

From the client, everything works exactly the same as traditional session-based authorization. You can Principal to retrieve objects from the request, check the role of the user, and other things using express-based access controls and @preauthorize annotations.

Each client in PiggyMetrics (account service, statistics service, notification service, and browser) has a scope: Server for back-end service, and UI – for browser. Therefore, we can also protect the controller from external access, for example:

@PreAuthorize("#oauth2.hasScope('server')")
@RequestMapping(value = "accounts/{name}", method = RequestMethod.GET)
public List<DataPoint> getStatisticsByAccountName(@PathVariable String name) {
    return statisticsService.findByAccountName(name);
}Copy the code

The API Gateway can see that there are three core services that expose external apis to clients. In real-world systems, this number can grow rapidly as well as the overall complexity of the system. In fact, hundreds of services may involve rendering a complex web page.

In theory, a client could make a request directly to each microserver. Obviously, however, this option has challenges and limitations, such as the need to know all endpoint addresses, perform HTTP requests for each information separately, and merge the results on the client side. Another issue is non-Web-friendly protocols that may be used on the back end.

Usually a better approach is to use an API gateway. It is a single entry point into the system to process requests by routing them to the appropriate back-end services, or by calling multiple back-end services and aggregating the results. In addition, it can be used for authentication, insight, stress and canary testing, service migration, static response processing, and active traffic management.

Netflix turned on such an edge service, and now with Spring Cloud, we can enable it with an @enableZuulProxy annotation. In this project, I used Zuul to store static content (UI applications) and route requests to the appropriate microservices. This is a simple prefix based route configuration Notification service:

Zuul: routes: notification-service: path: / notifications / ** serviceId: notification-service stripPrefix: falseCopy the code

This means that all incoming requests /notifications will be routed to the Notification service. There’s no hard-coded address, as you can see. Zuul uses the service discovery mechanism to locate notification service instances, as well as circuit breakers and load balancers, as described below.

Another well-known architectural pattern for Service Discovery is Service discovery. It allows automatic detection of the network location of service instances, which may dynamically assign addresses due to automatic scaling, failures, and upgrades.

A key part of service discovery is the registry. I used Netflix Eureka for this project. Eureka is a good example of the client-side discovery pattern when the client is responsible for determining the location of available service instances (using a registry server) and load balancing requests.

With Spring Boot, you can easily build the Eureka registry using spring-Cloud-starter-Eureka-Server dependencies, @enableEurekaserver annotations, and simple configuration properties.

Clients that support the @enableDiscoveryClient annotation support bootstrap.yml application name:

Spring: Application: name: notification-serviceCopy the code

Now, when the application starts, it registers with the Eureka server and provides metadata such as host and port, health indicator URL, home page, and so on. Eureka receives heartbeat messages from each instance of a service. If heartbeat failover occurs to a configurable schedule, the instance is removed from the registry.

In addition, Eureka provides a simple interface where you can keep track of running services and the number of available instances: http://localhost:8761

Load balancer, breaker and Http client Netflix OSS provides another great toolset.

Ribbon Ribbon is a client load balancer that gives you a lot of control over the behavior of HTTP and TCP clients. Compared to traditional load balancers, there are no additional hops per online call – you can directly contact the required services.

Out of the box, it integrates with Spring Cloud and service discovery itself. The Eureka Client provides a dynamic list of available servers so that the Ribbon can balance them.

Hystrix Hystrix is an implementation of the circuit breaker mode, which provides delay and failure control over dependency on access over the network. The main idea is to stop cascading failures in a distributed environment with a large number of microservices. This helps to fail quickly and recover as quickly as possible – an important aspect of self-healing fault tolerant systems.

In addition to circuit breaker control, with Hystrix, you can add a fallback method that can be called to get default values if the main command fails.

In addition, Hystrix generates metrics of execution results and latency for each command, which we can use to monitor system behavior.

Feign Feign is a declarative Http client that integrates seamlessly with the Ribbon and Hystrix. In fact, with a spring-cloud-starter-Feign dependency and the @enableFeignClients annotation, you have a complete set of load balancers, circuit breakers, and Http clients with a reasonable out-of-the-box default configuration.

Here is an example of an account service:

@FeignClient(name = "statistics-service") public interface StatisticsServiceClient { @RequestMapping(method = RequestMethod.PUT, value = "/statistics/{accountName}", consumes = MediaType.APPLICATION_JSON_UTF8_VALUE) void updateStatistics(@PathVariable("accountName") String accountName,  Account account); }Copy the code

• All you need is an interface
• You can use @requestMapping to share parts between the Spring MVC controller and Feign methods
• The above example specifies that only the service id-statistics-service is needed, since it is automatically discovered through Eureka (but obviously, you can access any resource with a specific URL)

Monitoring Dashboards In this project configuration, each Hystrix microservice pushes metrics to Turbine via The Spring Cloud Bus (using an AMQP agent). The monitor project is just a small Spring startup application with turbo and Hystrix dashboards.

Here’s how to make it work.

Let’s look at how our system behaves under load: The account service invokes the statistical service, which responds with a different impersonation delay. The response timeout threshold is set to 1 second.

0 ms delay	500 ms delay	800 ms delay	1100 ms delay
A system that performs well. Throughput is about 22 requests/second. The number of active threads in the statistics service is low. The median service time is about 50 ms.	The number of active threads is increasing. We can see the number of purple thread pool rejects, so about 30-40% error, but the circuit is still closed.	Half-open: When the fault command ratio is greater than 50%, the circuit breaker is on. After the time of the sleep window, the next request is allowed through.	100% of requests fail. The circuit is now permanently open. Retry after sleep time does not close the circuit again because a single request is too slow.

Log analysis Centralized logging can be very useful when trying to identify problems in a distributed environment. Elasticsearch, Logstash, and Kibana stacks make it easy to search and analyze your log, utilization, and network activity data. The out-of-the-box Docker configuration described in my other projects.

Security Advanced security configurations are outside the scope of this proof-of-concept project. For a more realistic simulation of a real system, consider using HTTPS, JCE keystore to encrypt microservice passwords and configure server property content (see documentation for more information).