preface

Hello everyone, today we start sharing Dubbo special Dubbo SPI. In the previous section, we discussed Dubbo service online testing and learned about the basic use of service testing and how it is implemented: the core principle is to make remote calls through metadata and using the GenericService API without relying on interface JAR packages. That chapter focuses on the SPI extension mechanism in Dubbo. What is SPI? And how does that play into our project? So let’s discuss that in this chapter. Let’s get started quickly!

1. Introduction to Dubbo SPI

What is Dubbo SPI? Its essence is enhanced from the STANDARD Service Provider Interface (SPI) extension point discovery mechanism of JDK. Dubbo improves on the following issues with the JDK standard SPI:

  1. The JDK’s standard SPI instantiates all implementations of extension points at once, which is time-consuming to initialize if there is an extension implementation, but wasteful of resources to load without it.

  2. If the extension point fails to load, you will not even get the name of the extension point. Such as: If the RubyScriptEngine class fails to load because the jruby.jar on which RubyScriptEngine depends does not exist, the class fails to load because the jruby.jar on which RubyScriptEngine depends does not exist. This failure cause is eaten up and does not correspond to Ruby. When a user executes a Ruby script, it will report that Ruby is not supported, rather than the actual failure cause.

  3. Added support for extension points IoC and AOP, where one extension point can directly setter for injection of other extension points.

To summarize, SPI on-demand loading in Dubbo saves resources, fixes Java SPI being ignored due to class loading failures, and adds support for IoC and AOP.

Dubbo SPI supports the following extensions:

  1. Protocol extensions

  2. Call interception extension

  3. Reference listening extension

  4. Exposure monitor extension

  5. The cluster expansion

  6. Routing extension

  7. Load Balancing expansion

  8. Merge result extension

  9. Registry Expansion

  10. Monitoring Center Expansion

  11. Extension points load extensions

  12. Dynamic proxy extension

  13. Compiler extension

  14. Dubbo configuration center extension

  15. Message dispatch extension

  16. Thread pool extension

  17. Serialized extension

  18. Network transmission extension

  19. Information exchange extension

  20. Network extension

  21. Telnet Command Extension

  22. Status check extension

  23. Container extension

  24. Cache extension

  25. Verify the extension

  26. Log Adaptation Extension

2. Usage

There are three paths in Dubbo to store these extended configurations: META-INF/dubbo, META-INF/dubbo/internal, META-INF/services/ the second directory is used to store the SPI extension inside Dubbo. The first and third directories are available to us. The content of the extension file is: Configuration name = fully qualified name of the extension implementation class. Multiple implementation classes are separated by newline characters. The file name is fully qualified name of the class. Such as:

| – resources

| – meta-inf

| – dubbo

​ | – org.apache.dubbo.rpc.Filter

​ | – custom=com.muke.dubbocourse.spi.custom.CustomFilter

META-INF/services/ is the SPI path provided by the JDK.

3. Application scenarios

Dubbo’s extensibility point is one of the reasons Dubbo is one of the most popular RPC frameworks. It takes flexibility and extensibility to the extreme. This was useful when we were customizing the Dubbo framework, where we performed simple extensions and configurations to achieve powerful functionality. Here are some common scenarios that are used in everyday work:

  1. Log printing: Print the parameter log when the service method call enters, and print the parameter log before the method call returns.

  2. Performance monitoring: It takes time to log the entire method call before it is finished and returns.

  3. Link tracing: Pass the call trace ID of each system in the Dubbo RPC call link, and monitor the link by integrating other link tracing systems.

4. Example demonstration

Here we also use an example to get a list of books, and we define a Filter to output logs to us before and after the service is called. The project structure is as follows:

In the above structure we customize the CustomFilter code as follows:

/ * * *@author<a href="http://youngitman.tech"> Young IT male </a> *@versionV1.0.0 *@className CustomFilter
 * @descriptionCustom filters *@JunitTest: {@link  }
 * @dateThe 2020-12-06 "* * /
public class CustomFilter implements Filter {
    @Override
    public Result invoke(Invoker
        invoker, Invocation invocation) throws RpcException {

        System.out.println("Custom filter before execution");

        Result result = invoker.invoke(invocation);

        System.out.println("Custom filter after execution");

        returnresult; }}Copy the code

We implement Filter and print the log output before and after Invoker is called. Let’s look at the dubo-provider-xml.xml configuration on the service provider side:


      
<beans xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:dubbo="http://dubbo.apache.org/schema/dubbo"
       xmlns="http://www.springframework.org/schema/beans"
       xsi:schemaLocation="Http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-4.3.xsd http://dubbo.apache.org/schema/dubbo http://dubbo.apache.org/schema/dubbo/dubbo.xsd">

    <dubbo:protocol port="20880" />

    <! Filetr key = custom -->
    <dubbo:provider filter="custom"/>

    <dubbo:application name="demo-provider" metadata-type="remote"/>

    <dubbo:registry address=Zookeeper: / / "127.0.0.1:2181"/>

    <bean id="bookFacade" class="com.muke.dubbocourse.spi.provider.BookFacadeImpl"/>

    <! -- Exposed service for Dubbo service -->
    <dubbo:service interface="com.muke.dubbocourse.common.api.BookFacade" ref="bookFacade" />

</beans>
Copy the code

In the previous configuration, we configured
to specify custom filters. . Then we in the resources – > meta-inf dubbo directory new org. Apache. Dubbo. RPC. The Filter configuration file is as follows:

custom=com.muke.dubbocourse.spi.custom.CustomFilter
Copy the code

Where custom for our extension key is consistent with our configuration in XML.

5. Principle analysis

Dubbo in the SPI extension load using ExtensionLoader below we simply through the source code to analyze. First entry for static function org.apache.dubbo.com mon. The extension. ExtensionLoader# ExtensionLoader code is as follows:

    private ExtensionLoader(Class
        type) {
        // The extended class type to load
        this.type = type;
        // The container factory executes the ExtensionFactory load first and then getAdaptiveExtension if the ExtensionFactory object is not loaded
        objectFactory = (type == ExtensionFactory.class ? null : ExtensionLoader.getExtensionLoader(ExtensionFactory.class).getAdaptiveExtension());
    }
Copy the code

The above method is simply to get the ExtensionLoader object, but it is worth noting that there is a layer of recursive calls until the loading type is ExtensionFactory. Let’s look at the getAdaptiveExtension code:

    public T getAdaptiveExtension(a) {
        // Cache fetch
        Object instance = cachedAdaptiveInstance.get();
        if (instance == null) {
           / /...
            // Determine the lock
            synchronized (cachedAdaptiveInstance) {
                // Get double lock detection again
                instance = cachedAdaptiveInstance.get();
                if (instance == null) {
                    try {
                        // Create an extended instance
                        instance = createAdaptiveExtension();
                        // Cache
                        cachedAdaptiveInstance.set(instance);
                    } catch (Throwable t) {
                     / /...}}}}return (T) instance;
    }
Copy the code

Let’s look at how the createAdaptiveExtension method creates an instance:

    private T createAdaptiveExtension(a) {
        try {
            // First create the extended instance, then inject the dependency
            return injectExtension((T) getAdaptiveExtensionClass().newInstance());
        } catch (Exception e) {
            throw new IllegalStateException("Can't create adaptive extension " + type + ", cause: "+ e.getMessage(), e); }}Copy the code

GetAdaptiveExtensionClass method code is as follows:

    privateClass<? > getAdaptiveExtensionClass() {// Get the extension class
        getExtensionClasses();
        if(cachedAdaptiveClass ! =null) {
            return cachedAdaptiveClass;
        }
        // Dynamically generate Class
        return cachedAdaptiveClass = createAdaptiveExtensionClass();
    }
Copy the code

GetExtensionClasses core code is as follows:

 /*** ** get all extensions to Class **@author liyong
     * @datePrepare the 2020-02-27 *@param
     * @exception
     * @returnjava.util.Map<java.lang.String,java.lang.Class<? > > * * /
    privateMap<String, Class<? >> getExtensionClasses() { Map<String, Class<? >> classes = cachedClasses.get();if (classes == null) {
            synchronized (cachedClasses) {
                classes = cachedClasses.get();
                if (classes == null) {
                    // Start loading Class from the resource path
                    classes = loadExtensionClasses();
                   // Set the cachecachedClasses.set(classes); }}}return classes;
    }
Copy the code

The loadExtensionClasses code looks like this:

    /** * * CLASS_PATH=org.apache.dubbo.common.extension.ExtensionFactory * * 1.META-INF/dubbo/internal/${CLASS_PATH} META-INF/ Dubbo /${CLASS_PATH} User-defined extension path * 3.META-INF/services/{CLASS_PATH} JdkSPI path * * synchronized in getExtensionClasses * */
    privateMap<String, Class<? >> loadExtensionClasses() { cacheDefaultExtensionName(); Map<String, Class<? >> extensionClasses =new HashMap<>();
        // internal extension load from ExtensionLoader's ClassLoader first
        loadDirectory(extensionClasses, DUBBO_INTERNAL_DIRECTORY, type.getName(), true);
        // Compatibility handling due to dubbo donation to Apache
        loadDirectory(extensionClasses, DUBBO_INTERNAL_DIRECTORY, type.getName().replace("org.apache"."com.alibaba"), true);

        loadDirectory(extensionClasses, DUBBO_DIRECTORY, type.getName());
        loadDirectory(extensionClasses, DUBBO_DIRECTORY, type.getName().replace("org.apache"."com.alibaba"));
        loadDirectory(extensionClasses, SERVICES_DIRECTORY, type.getName());
        loadDirectory(extensionClasses, SERVICES_DIRECTORY, type.getName().replace("org.apache"."com.alibaba"));
        return extensionClasses;
    }

Copy the code

Next we see the loadDirectory method that actually loads the resource:

 private void loadDirectory(Map<String, Class<? >> extensionClasses, String dir, String type,boolean extensionLoaderClassLoaderFirst) {
        String fileName = dir + type;
        try {
            Enumeration<java.net.URL> urls = null;
            ClassLoader classLoader = findClassLoader();
            
            // try to load from ExtensionLoader's ClassLoader first
            if (extensionLoaderClassLoaderFirst) {
                ClassLoader extensionLoaderClassLoader = ExtensionLoader.class.getClassLoader();
                // The class loader of ExtensionLoader is used first, probably by user custom loading
                if (ClassLoader.getSystemClassLoader() != extensionLoaderClassLoader) {
                    urls = extensionLoaderClassLoader.getResources(fileName);
                }
            }
            
            if(urls == null| |! urls.hasMoreElements()) {if(classLoader ! =null) {// Use AppClassLoader to load
                    urls = classLoader.getResources(fileName);
                } else{ urls = ClassLoader.getSystemResources(fileName); }}if(urls ! =null) {
                while (urls.hasMoreElements()) {
                    java.net.URL resourceURL = urls.nextElement();
                    // Load resourcesloadResource(extensionClasses, classLoader, resourceURL); }}}catch (Throwable t) {
            logger.error("Exception occurred when loading extension class (interface: " +
                    type + ", description file: " + fileName + ").", t); }}Copy the code

LoadResource = loadResource; loadResource = loadResource

    // Load file values to Class to Map
    private void loadResource(Map
       
        > extensionClasses, ClassLoader classLoader, java.net.URL resourceURL)
       ,> {
        try {
            try (BufferedReader reader = new BufferedReader(new InputStreamReader(resourceURL.openStream(), StandardCharsets.UTF_8))) {
                String line;
                // Read a line of data
                while((line = reader.readLine()) ! =null) {
                    final int ci = line.indexOf(The '#');// Remove the comment
                    if (ci >= 0) {
                        line = line.substring(0, ci);
                    }
                    line = line.trim();
                    if (line.length() > 0) {
                        try {
                            String name = null;
                            int i = line.indexOf('=');
                            if (i > 0) {
                                name = line.substring(0, i).trim();
                                line = line.substring(i + 1).trim();
                            }
                            if (line.length() > 0) {
                                / / load the Class
                                loadClass(extensionClasses, resourceURL, Class.forName(line, true, classLoader), name); }}catch (Throwable t) {
                           / /...
                        }
                    }
                }
            }
        } catch (Throwable t) {
           / /...}}Copy the code

The above method loops through each row of data and resolves the path after = to load the Class. This loop loads all classes configured through the configuration file under the custom resource path.

6. Summary

In this section, we mainly learn the SPI in Dubbo. First of all, we know that the essence of Dubbo SPI is strengthened from the Service Provider Interface (SPI) extension point discovery mechanism of JDK standard. It also addresses some of the shortcomings of SPI in Java. We also use simple use cases to explain how to extend in our daily work, and to explain the loading principle of SPI from a source point of view. The core entry class is ExtensionLoader.

The highlights of this lesson are as follows:

  • Understand SPI in Dubbo
  • Understand the differences between Dubbo SPI and Java SPI
  • See how Dubbo is using SPI to expand
  • Understand SPI implementation principles

The author

Personally engaged in the financial industry, I have worked in chongqing’s first-class technical team of Yiji Pay, Sijian Technology and an online car hailing platform, and now I am working in a bank responsible for the construction of unified payment system. I have a strong interest in the financial industry. It also practices big data, data storage, automated integration and deployment, distributed microservices, responsive programming, and artificial intelligence. At the same time, he is also keen on technology sharing, creating public accounts and blog sites to share knowledge system. Concern public number: young IT male get latest technical article push!

Blog: Youngitman.tech

Wechat Official Account: