This is the 27th day of my participation in the August More Text Challenge

The premise is introduced

  • If you want to design and develop a microservice infrastructure, parameterization is a very important point, and Netflix has opened source a configuration center client called Chameleon Archaius, and Archaius is arguably more production-level features and more flexible than other clients.

  • In The NetflixOSS microservice stack, almost all other components (such as Zuul, Hystrix, Eureka, Ribbon, etc.) rely on Archaius. It can be said that understanding Archaius is the basis for understanding and using other Netflix microservice components.

What is Archaius

Netflix Archaius is a configuration management library that focuses on dynamic properties from multiple configuration stores. It includes a set of Java configuration management apis for Netflix. It is implemented primarily as an extension of the Apache Commons Configuration library. The main functions provided are:

Note that Netflix has only open-source the client part of its configuration center (aka Archaius), not the server side. Archaius is actually configuration source implementation independent and can be connected to various configuration centers as data sources. Later in this article, we will show how Archaius integrates with the Apollo configuration center.

The origin of Archaius

  • In a microservice environment, the configuration often needs to be adjusted for different contexts, or the configuration should be multi-dimensional. In Netflix, for example, the context dimension includes the environment (development, test, and production).

  • Netflix hopes to dynamically adjust the service configuration according to the publishing environment or even the context of the request, so that the behavior and logic of the entire Netflix system can be dynamically adjusted to meet the fast and changeable demands of Internet applications. To this end, the Netflix platform team developed a configuration center product, which they aptly named Archaius, a chameleon, because of its ability to dynamically adjust its body color based on its environment.

Archaius in Netflix use case scenario

  1. Turn a feature on or off, depending on the context of the request.
  2. By default, 10 items are displayed on a page, and in some cases, you can adjust the configuration using Archaius to show only five items.
  3. Dynamically adjust Hystrix fuse behavior.
  4. Adjust connection and request timeout parameters for the service invocation client.
  5. If an error alarm is generated for an online service, you can dynamically adjust the log output level (the granularity can be as small as the package or component level) to troubleshoot the fault using detailed logs. After the fault is located, restore the log output level to the default level.
  6. For applications deployed in multiple regions or countries, dynamic configuration enables different functions for different regions and countries.
  7. The configuration of some basic middleware can be dynamically adjusted according To the actual access mode of users, such as the Time To Live (TTL) of cache.
  8. The database access client connection pool configuration allows different values for different services. For example, a service with a small Request Per Second (RPS) frequency can be configured with a small number of connections, while a service with a large Request frequency can be configured with a large number of connections.
  9. Run-time configuration changes can take effect in different dimensions, such as a single instance dimension in a cluster, a region dimension in a multi-region deployment, a service stack dimension, or an application cluster dimension.
  10. Feature flags are released. Although some functions are online, they are not immediately enabled, but dynamically enabled by configuring the switch, so that a certain online function can be flexibly enabled or disabled according to the situation.
  11. Canary Release. When the new function is released, the new and old clusters coexist for a period of time, and the traffic to the old cluster is gradually dynamically adjusted to the new cluster through configuration. If the monitoring shows no abnormality, the new cluster is launched.

Technical basis of Archaius

  • Archaius is Netflix’s open source dynamic property configuration framework. Based on Apache Commons Configuration, archaius provides the function of obtaining configuration values at run time.

  • At the heart of Archaius is the concept of composite configurations that can accommodate one or more configurations. Each configuration can be obtained from configuration sources such as JDBC, REST interfaces, and xxx.properties files. Optionally polling the configuration source at run time for dynamic changes,

  • The final value of the property depends on the topmost configuration that contains the property (because it is a composite configuration). That is, if a property exists in more than one configuration, the actual value the application sees will be the value in the top-level slot in the configuration hierarchy, which of course can be configured.

Architectural design of Archaius

Archaius is a package and extension of the Apache Common Configuration Library that provides a set of Java-based Configuration apis. The main features include:

  • Configuration can be dynamically adjusted: dynamic, type properties
  • The configuration supports types (Int, Long, Boolean, etc.).
  • High performance and thread-safety: High throughput and thread-safe configuration operations
  • Provides a framework for pulling configurations that dynamically pull changed configurations from configuration sources. (A polling framework that allows users to get property changes to configuration sources)
  • Support for callback mechanism, automatically called when configuration changes.
  • JMX MBeans are supported, and configurations can be viewed and modified through JConsole.

For applications (and most Web applications) that are willing to use convention-based properties file locations, out-of-the-box composite configuration (one of the most powerful features) is provided, as shown in an example diagram from the conformance configuration website:

At the heart of Achaius is a concept called Composite Configuration, which can be simply understood as a hierarchy of configurations. The hierarchy has a priority, and the Configuration of a higher priority overrides the Configuration of a lower priority. Each tier can obtain configuration from a configuration source, such as a local configuration file, JDBC data source, remote REST API, and so on. Configuration sources can also pull changes dynamically at runtime, for example in the illustration above, Persisted DB Configuration, which is the Configuration that is stored in a relational database and that is pulled from the database periodically. The final value of the configuration is determined by the top-level configuration. For example, if multiple levels contain a configuration item, the value that the application eventually sees is the top-level value in the configuration level. The order of configuration layering can be adjusted.

There are two ways to get configuration values from archaius:
  • One is to get the configuration center instance through the ConfigurationManager, then get the configuration value through the propName, and then get the configuration through the ConfigurationManager

  • Another way is to get the DynamicProperty Wrapper for the configuration item via DynamicPropertyFactory.

Implementation principle of Archaius

What is Archaius?

Archaius provides the function of dynamically changing the value of the configuration without restarting the application service. The core idea is to poll the configuration source, detect whether the configuration has changed, and update the configuration again with each iteration.

The underlying Archaius provides a concrete implementation of AbstractConfiguration that implements Apache-common-Configuration.

  • ConcurrentMapConfiguration provide the configuration of configuration values in a ConcurrentHashMap maintenance function.

  • DynamicConfiguration provides the ability to dynamically fetch all configuration values from the data source and poll the data source to update the configuration values.

  • DynamicWatchedConfiguration also provide dynamic update configuration function, unlike DynamicConfiguration, configuration updates are triggered when data are changed.

  • DynamicConfiguration is pull way, DynamicWatchedConfiguration is push way.

ConcurrentCompositeConfiguration mode, using a combination of the combination of different AbstractConfiguration implementation. Have multiple configuration source for the configuration of the center, you can use ConcurrentCompositeConfiguration. If multiple configuration sources have configuration values for the same configuration item, obtain the data from the first matched configuration source.

How does DynamicPropertyFactory work?

  • DynamicPropertyFactory holds an AbstractConfiguration instance.

  • When a DynamicProperty object is created, the DynamicProperty object gets an AbstractConfiguration instance held by DynamicPropertyFactory.

  • AbstractConfiguration instance DynamicProperty register DynamicPropertyListener. AbstractConfiguration instance DynamicProperty register DynamicPropertyListener. The current DynamicProperty object is notified.

This can be a performance issue when creating a large number of DynamicProperty instances. Each time a DynamicProperty is created, a listener is added, and a listener is triggered when any configuration item changes.

Probably because there won’t be as many configuration changes in production. Watcher is better triggered by configuration item changes like zK-config.

How is AbstractConfiguration injected?

Archaius allows only one AbstractConfiguration implementation class. If there are multiple configuration source, you can use the above mentioned ConcurrentCompositeConfiguration AbstractConfiguration together will be different.

There are several ways to inject AbstractConfiguration.

  1. Configuration archaius. Default. The configuration. The class specifies AbstractConfiguration implementation class

  2. Configuration archaius. Default. The configuration. The factory designated AbstractConfiguration class instance factory methods, factory method class needs to implement the getInstance () method returns AbstractConfiguration instance

  3. Call the Install (AbstractConfiguration Config) method of the ConfigurationManager

  4. Call DynamicPropertyFactory initWithConfigurationSource (AbstractConfiguration config) method

The above methods are mutually exclusive. Only one of them can be used. When one of them takes effect, the others cannot be called.

A simple example:
Maven dependency Configuration
<dependency>
    <groupId>com.netflix.archaius</groupId>
    <artifactId>archaius-core</artifactId>
    <version>0.7.7</version>
</dependency>
Copy the code
Obtaining configuration Sources
/** * Created by longfei on 17/1/19. */
public class DynamicConfigurationSource implements PolledConfigurationSource {
    @Override
    public PollResult poll(boolean initial,Object checkPoint) throws Exception {
        Map<String,Object> map = new HashMap<>();
        map.put("test",UUID.randomUUID().toString());
        returnPollResult.createFull(map); }}Copy the code
Define the scheduler
AbstractPollingScheduler scheduler = new FixedDelayPollingScheduler(2000.2000.false);
Copy the code
Defining dynamic configuration
DynamicConfiguration configuration = new DynamicConfiguration(source,scheduler);
Copy the code
Simple unit tests
@org.testng.annotations.Test
    public void testArchaius(a) throws Exception {
        PolledConfigurationSource source = new DynamicConfigurationSource();
        AbstractPollingScheduler scheduler = new FixedDelayPollingScheduler(2000.2000.false);
        DynamicConfiguration configuration = new DynamicConfiguration(source,scheduler);
        ConfigurationManager.install(configuration);
        final DynamicStringProperty stringProperty = DynamicPropertyFactory.getInstance().getStringProperty("test"."nodata");
        Helpers.subscribePrint(Observable.interval(1,TimeUnit.SECONDS).take(20).doOnNext(new Action1<Long>() {
                    @Override
                    public void call(Long aLong) { System.out.println(stringProperty.get()); }}),"test");
        TimeUnit.MINUTES.sleep(1);
    }
Copy the code

implementation

Example Start a polling task
public synchronized void startPolling(PolledConfigurationSource source, AbstractPollingScheduler scheduler) {
    this.scheduler = scheduler;
    this.source = source;
    init(source, scheduler);
    scheduler.startPolling(source, this);
}
Copy the code
Runnable and initialization of polling: implementation is consistent
PollResult result = null;
    try {
       result = source.poll(false,getNextCheckPoint(checkPoint));
       checkPoint = result.getCheckPoint();
       fireEvent(EventType.POLL_SUCCESS, result, null);
       } catch (Throwable e) {
       log.error("Error getting result from polling source", e);
       fireEvent(EventType.POLL_FAILURE, null, e);
       return;
       }
       try {
          populateProperties(result, config);
       } catch (Throwable e) {
           log.error("Error occured applying properties", e);
      }
Copy the code

Noticed that will invoke the source. The poll method, namely PolledConfigurationSource polled, we implement the data interface, can customize the data source (JDBC, documents, SCM, etc.)

conclusion

One of the biggest obstacles to understanding Archaius is Apache Commons Configuration, which relies heavily on the latter for Configuration management. How well you know Apache Commons Configuration determines how well you know Netflix Archaius.

Data reference

  • Github.com/netflix/arc…
  • Techblog.ppdai.com/2018/05/08/…