Note: This source code analysis corresponds to JDK version 1.8
1 the introduction
This is [source notes] JDK source code interpretation of the first article, this we will explore the Java SPI mechanism of the relevant source.
What is SPI mechanism
So, what is the SPI mechanism?
SPI stands for Service Provider Interface, which stands for Service Provider Interface. SPI is basically an extension mechanism. We define an interface implementation class in the corresponding configuration file, and then load the instance class into the configuration file based on that interface and instantiate it. That’s what SPI is. When it comes to SPI mechanisms, the most common is Java’s SPI mechanism, in addition to Dubbo and SpringBoot’s custom SPI mechanism.
With the SPI mechanism, it is possible to extend some framework flexibly without having to write some implementation classes of the framework in code.
So how do some frameworks take advantage of the SPI mechanism to extend flexibly? Here are a few chestnuts to illustrate:
- JDBC driver loading case: using Java SPI mechanism, we can introduce different JDBC driver packages according to different database vendors;
- SpringBoot’s SPI mechanismB: We can do it at
spring.factoriesAdd our custom automatic configuration classes, event listeners or initializers, etc. - SPI mechanism of Dubbo: Dubbo is the application of SPI mechanism incinctly. Basically, every function point of Dubbo itself provides extension points, such as cluster extension, routing extension and load balancing extension, which are nearly 30 extension points. If one of Dubbo’s built-in implementations doesn’t meet our needs, we can simply replace our implementation with Dubbo’s using its SPI mechanism.
Let’s first get a feel for how some frameworks can extend flexibly using the SPI mechanism.
3 How to use Java SPI?
Let’s start by looking at how to use Java’s own SPI. Let’s define a Developer interface
// Developer.java
package com.ymbj.spi;
public interface Developer {
void sayHi(a);
}
Copy the codeDefine two more implementation classes for the two Developer interfaces:
// JavaDeveloper.java
package com.ymbj.spi;
public class JavaDeveloper implements Developer {
@Override
public void sayHi(a) {
System.out.println("Hi, I am a Java Developer."); }}Copy the code// PythonDeveloper.java
package com.ymbj.spi;
public class PythonDeveloper implements Developer {
@Override
public void sayHi(a) {
System.out.println("Hi, I am a Python Developer."); }}Copy the codeCreate a meta-INF /services folder in the resources directory of the project, and create a new file with the fully qualified name of the Developer interface:
/ / com. Ymbj. Spi. The Developer documentation
com.ymbj.spi.JavaDeveloper
com.ymbj.spi.PythonDeveloper
Copy the codeFinally, create a new test class, JdkSPITest:
// JdkSPITest.java
public class JdkSPITest {
@Test
public void testSayHi(a) throws Exception { ServiceLoader<Developer> serviceLoader = ServiceLoader.load(Developer.class); serviceLoader.forEach(Developer::sayHi); }}Copy the codeRun the above test class and the successful result is shown in the screenshot below:
From the simple Demo above, we know how to use Java’s SPI mechanism to implement extension point loading. The following article is recommended: Java’s SPI mechanism, through this article, I believe you will have a deep understanding of Java’s SPI, especially the JDBC loading driver.
4 Java SPI mechanism source interpretation
The ServiceLoader class is a ServiceLoader class. The ServiceLoader class is a ServiceLoader class.
ServiceLoader implements the Iterable interface
public final class ServiceLoader<S>
implements 可迭代<S>{
private static final String PREFIX = "META-INF/services/";
// The class or interface representing the service being loaded
private final Class<S> service;
// The class loader used to locate, load, and instantiate providers
private final ClassLoader loader;
// The access control context taken when the ServiceLoader is created
private final AccessControlContext acc;
// Cached providers, in instantiation order
private LinkedHashMap<String,S> providers = new LinkedHashMap<>();
// The current lazy-lookup iterator
private LazyIterator lookupIterator;
// constructor
private ServiceLoader(Class<S> svc, ClassLoader cl) {
service = Objects.requireNonNull(svc, "Service interface cannot be null");
loader = (cl == null) ? ClassLoader.getSystemClassLoader() : cl;
acc = (System.getSecurityManager() != null)? AccessController.getContext() :null;
reload();
}
/ /... Omit the relevant code for now
// The ServiceLoader inner class LazyIterator implements the Iterator interface
// Private inner class implementing fully-lazy provider lookup
private class LazyIterator
implements Iterator<S>{
Class<S> service;
ClassLoader loader;
Enumeration<URL> configs = null;
Iterator<String> pending = null;
String nextName = null;
private LazyIterator(Class<S> service, ClassLoader loader) {
this.service = service;
this.loader = loader;
}
// Override the hasNext method of the Iterator interface
public boolean hasNext(a) {
/ /... Omit the relevant code for now
}
// Overrides the next method of the Iterator interface
public S next(a) {
/ /... Omit the relevant code for now
}
// Override the remove method of the Iterator interface
public void remove(a) {
/ /... Omit the relevant code for now}}// Overrides the iterator method of the Iterable interface, returning an iterator
public Iterator<S> iterator(a) {
/ /... Omit the relevant code for now
}
/ /... Omit the relevant code for now
}
Copy the codeServiceLoader implements the Iterable interface. Overwriting its iterator method produces an iterator. LazyIterator is an Iterator. LazyIterator is an Iterator. LazyIterator is an Iterator.
4.1 Serviceloader. load method to prepare for loading the service provider implementation class
So let’s start exploring the source code of Java’s SPI mechanism, ServiceLoader
ServiceLoader = serviceloader.load (developer.class); The ServiceLoader. Load (Developer. Class); Source:
// ServiceLoader.java
public static <S> ServiceLoader<S> load(Class<S> service) {
// Get the current thread context classloader
ClassLoader cl = Thread.currentThread().getContextClassLoader();
// Pass in the Service interface class and the thread context classloader as arguments and proceed with the load method
return ServiceLoader.load(service, cl);
}
Copy the codeSo serviceloader.load (service, cl); Methods:
// ServiceLoader.java
public static <S> ServiceLoader<S> load(Class service, ClassLoader loader)
{
// Create a ServiceLoader object with the Service interface class and thread context classloader as construction parameters
return new ServiceLoader<>(service, loader);
}
Copy the codeNew ServiceLoader<>(service, loader); How is it built?
// ServiceLoader.java
private ServiceLoader(Class<S> svc, ClassLoader cl) {
service = Objects.requireNonNull(svc, "Service interface cannot be null");
loader = (cl == null) ? ClassLoader.getSystemClassLoader() : cl;
acc = (System.getSecurityManager() != null)? AccessController.getContext() :null;
reload();
}
Copy the codeYou can see that the ServiceLoader object is built using the reload method in addition to assigning its member attributes:
// ServiceLoader.java
public void reload(a) {
providers.clear();
lookupIterator = new LazyIterator(service, loader);
}
Copy the codeYou can see that a new LazyIterator object is created in the Reload method and then assigned to lookupIterator.
// ServiceLoader$LazyIterator.java
private LazyIterator(Class<S> service, ClassLoader loader) {
this.service = service;
this.loader = loader;
}
Copy the codeThe LazyIterator object is a meta-INF /services folder. The LazyIterator object is a meta-INF /services folder. This is weird because we’ve all been fooled by the ServiceLoader load method name.
Remember when we analyzed the previous code and created a new LazyIterator object? Lazy is an Iterator. We assume that the purpose of the LazyIterator is to load the implementation class of the Developer interface during iteration.
4.2 Serviceloader. iterator method to implement lazy loading of service provider implementation classes
ServiceLoader. ForEach (Developer::sayHi); The iterator method of the serviceLoader will eventually be called after executing this code:
// serviceLoader.java
public Iterator<S> iterator(a) {
return new Iterator<S>() {
Iterator<Map.Entry<String,S>> knownProviders
= providers.entrySet().iterator();
public boolean hasNext(a) {
if (knownProviders.hasNext())
return true;
// Call lookupIterator, the hasNext method of LazyIterator
// Delegate to the hasNext method of LazyIterator
return lookupIterator.hasNext();
}
public S next(a) {
if (knownProviders.hasNext())
return knownProviders.next().getValue();
// Call lookupIterator, the next method of LazyIterator
// Delegate to the next method of LazyIterator
return lookupIterator.next();
}
public void remove(a) {
throw newUnsupportedOperationException(); }}; }Copy the codeThe serviceLoader iterator returns an anonymous iterator that is essentially a facade class whose overridden hasNext and Next methods delegate to LazyIterator’s hasNext and Next methods, respectively.
As we continue debugging, we find that we will enter LazyIterator’s hasNext method:
// serviceLoader$LazyIterator.java
public boolean hasNext(a) {
if (acc == null) {
Call the hasNextService method
return hasNextService();
} else {
PrivilegedAction<Boolean> action = new PrivilegedAction<Boolean>() {
public Boolean run(a) { returnhasNextService(); }};returnAccessController.doPrivileged(action, acc); }}Copy the codeFollow up with hasNextService methods:
// serviceLoader$LazyIterator.java
private boolean hasNextService(a) {
if(nextName ! =null) {
return true;
}
if (configs == null) {
try {
// PREFIX = "META-INF/services/"
// service.getName() is the fully qualified name of the interface
// Remember the previous code that built the LazyIterator with the service attribute assigned to it
String fullName = PREFIX + service.getName();
// Load the service provider class in the interface file in meta-INF /services/
if (loader == null)
configs = ClassLoader.getSystemResources(fullName);
else
// Remember when the previous code built the LazyIterator object and assigned a value to its member loader
configs = loader.getResources(fullName);
} catch (IOException x) {
fail(service, "Error locating configuration files", x); }}while ((pending == null) | |! pending.hasNext()) {if(! configs.hasMoreElements()) {return false;
}
// Return the service provider class in the interface file in meta-INF /services/ and assign it to pending
pending = parse(service, configs.nextElement());
}
// Then assign a fully qualified name to the member variable nextName of LazyIterator
nextName = pending.next();
return true;
}
Copy the codeYou can see that executing LazyIterator’s hasNextService method will eventually load the contents of the interface file in the meta-INF /services/ directory, that is, the fully qualified name of the service provider implementation class, It then takes the fully qualified name of a service provider implementation class and assigns it to the member variable nextName of LazyIterator. At this point, we can see that LazyIterator is really lazy loading, loading only when needed.
Ponder: why use lazy loading here? What is the idea of lazy loading? What are the benefits of lazy loading? Can you name any other examples of lazy loading?
Similarly, after executing LazyIterator’s hasNext method, we continue executing LazyIterator’s next method:
// serviceLoader$LazyIterator.java
public S next(a) {
if (acc == null) {
// Call the nextService method
return nextService();
} else {
PrivilegedAction<S> action = new PrivilegedAction<S>() {
public S run(a) { returnnextService(); }};returnAccessController.doPrivileged(action, acc); }}Copy the codeWe continue to follow the nextService approach:
// serviceLoader$LazyIterator.java
private S nextService(a) {
if(! hasNextService())throw new NoSuchElementException();
// Remember assigning nextName to the fully qualified name of the service provider implementation class in the hasNextService method
String cn = nextName;
nextName = null; Class<? > c =null;
try {
// [1] Load the service provider implementation class in classpath according to the fully qualified name of the class loader and service provider implementation class passed in
c = Class.forName(cn, false, loader);
} catch (ClassNotFoundException x) {
fail(service,
"Provider " + cn + " not found");
}
if(! service.isAssignableFrom(c)) { fail(service,"Provider " + cn + " not a subtype");
}
try {
// [2] instantiate the service provider implementation class just loaded and transform it
S p = service.cast(c.newInstance());
// [3] Finally put the instantiated service provider implementation class into the providers collection
providers.put(cn, p);
return p;
} catch (Throwable x) {
fail(service,
"Provider " + cn + " could not be instantiated",
x);
}
throw new Error(); // This cannot happen
}
Copy the codeAs you can see, LazyIterator’s nextService method ends up instantiating the previously loaded service provider implementation class into the providers collection and then calling the service provider implementation class’s methods (in this case, JavaDeveloper’s sayHi method). If the main function has a method that calls the service provider implementation class, the method will be called immediately after it is loaded. Then proceed to instantiate the next service provider class.
Design pattern: As you can see, Java’s SPI mechanism implementation code applies the iterator pattern. The iterator pattern hides the internal structure differences of various storage objects and provides a unified view to traverse each storage object (storage objects can be collections, arrays, etc.). Iterator is also an implementation of the Iterator pattern: At the same time, each Java collection class generally implements Iterable interface, implements its Iterator method to obtain the implementation class object of Iterator interface (generally for the collection inner class), and then uses the hasNext and next methods of the implementation class of iterator object to traverse the collection elements.
5 Interpreting the JDBC driver loading source code
The previous analysis of the Java SPI mechanism source code implementation, now we look at the Java SPI mechanism of the actual case application.
As we all know, JDBC driver loading is a typical application of Java’s SPI mechanism. JDBC mainly provides a set of interface specifications, and the API of this specification is implemented in the Java core library (Rt.jar), and different database manufacturers as long as they write the driver code in accordance with this SET of JDBC interface specifications, then they can use the Java language to connect to the database.
Java core library (Rt.jar) with JDBC Driver loading the most core interface and class are java.sql.Driver interface and java.sql.DriverManager class, where java.sql. DriverManager is used to manage the database driver class. Note that DriverManager has a registeredDrivers collection property, which is used to store the database driver class.
// DriverManager.java
// List of registered JDBC drivers
private final static CopyOnWriteArrayList<DriverInfo> registeredDrivers = new CopyOnWriteArrayList<>();
Copy the codeHere to load Mysql driver as an example to analyze the JDBC driver load source code.
After introducing the mysql-connector-Java dependency (version 5.1.47) in our project, the mysql driver implementation class file looks like the following figure:
DriverDriver class, respectivelycom.mysql.jdbc.Driverandcom.mysql.fabric.jdbc.FabricMySQLDriverBy default we usually only use the former.
5.1 Load Mysql driver class using Java SPI
So let’s explore how the JDBC driver loaded code is implemented.
Let’s start with a simple JDBC test code:
// JdbcTest.java
public class JdbcTest {
public static void main(String[] args) {
Connection connection = null;
Statement statement = null;
ResultSet rs = null;
try {
// Note: In the JDBC 4.0 specification, you can no longer write code that explicitly loads the database
// Class.forName("com.mysql.jdbc.Driver");
Mysql > connect to mysql;
/*************** ****************/
connection = DriverManager.getConnection("jdbc:mysql://localhost:3306/jdbc"."root"."123456");
// Create Statement
statement = connection.createStatement();
// Execute the query statement
rs = statement.executeQuery("select * from user");
// Iterate over the query result set
while(rs.next()){
String name = rs.getString("name"); System.out.println(name); }}catch(Exception e) {
e.printStackTrace();
} finally {
/ /... Omit the code that frees resources}}}Copy the codeWhen JdbcTest’s main function calls DriverManager’s getConnection method, the static code block of DriverManager must be executed before the getConnection method is executed. So let’s look at the static code block for DriverManager:
// DriverManager.java
static {
// Load the driver implementation class
loadInitialDrivers();
println("JDBC DriverManager initialized");
}
Copy the codeFollow up with the loadInitialDrivers code:
// DriverManager.java
private static void loadInitialDrivers(a) {
String drivers;
try {
drivers = AccessController.doPrivileged(new PrivilegedAction<String>() {
public String run(a) {
return System.getProperty("jdbc.drivers"); }}); }catch (Exception ex) {
drivers = null;
}
AccessController.doPrivileged(new PrivilegedAction<Void>() {
public Void run(a) {
// If this sounds familiar, yes, it is. We have executed the following two lines of code in the JdkSPITest code
The service provider implementation class is not actually loaded
// Instead instantiate a ServiceLoader object and a LazyIterator object for lazy loading
ServiceLoader<Driver> loadedDrivers = ServiceLoader.load(Driver.class);
/ / call the ServiceLoader iterator method, at the same time of iteration, and go to load and instantiate the meta-inf/services/Java SQL. The Driver file
. / / com. Mysql. JDBC Driver and com. The mysql. Fabric. JDBC. FabricMySQLDriver class two drivers
/**************** [mainline, focus] **********************/
Iterator<Driver> driversIterator = loadedDrivers.iterator();
try{
while(driversIterator.hasNext()) { driversIterator.next(); }}catch(Throwable t) {
// Do nothing
}
return null; }}); println("DriverManager.initialize: jdbc.drivers = " + drivers);
if (drivers == null || drivers.equals("")) {
return;
}
String[] driversList = drivers.split(":");
println("number of Drivers:" + driversList.length);
for (String aDriver : driversList) {
try {
println("DriverManager.Initialize: loading " + aDriver);
Class.forName(aDriver, true,
ClassLoader.getSystemClassLoader());
} catch (Exception ex) {
println("DriverManager.Initialize: load failed: "+ ex); }}}Copy the codeIn the above code, we can see that Mysql driver class loading is mainly implemented using Java SPI mechanism, that is, using ServiceLoader to load and instantiate Mysql driver class.
5.2 Registering the Mysql driver Class
So, the above code is only load and instantiate Mysql driver class, so how is the driver class registered in DriverManager registeredDrivers collection?
Com.mysql.jdbc.driver has a static code block in it. Instantiating the class must trigger the execution of the static code block. Let’s look at what this static code block does:
// com.mysql.jdbc.Driver.java
// Register ourselves with the DriverManager
static {
try {
// Register yourself into the registeredDrivers collection of DriverManager
java.sql.DriverManager.registerDriver(new Driver());
} catch (SQLException E) {
throw new RuntimeException("Can't register driver!"); }}Copy the codeMysql Driver com.mysql.jdbc.driver registers itself in the registeredDrivers collection of DriverManager when it executes its static code block during instantiation.
Ok, follow up with the registerDriver method on DriverManager:
// DriverManager.java
public static synchronized void registerDriver(java.sql.Driver driver)
throws SQLException {
// Continue calling the registerDriver method
registerDriver(driver, null);
}
public static synchronized void registerDriver(java.sql.Driver driver, DriverAction da)
throws SQLException {
/* Register the driver if it has not already been added to our list */
if(driver ! =null) {
// Register the driver class instance into the registeredDrivers collection
registeredDrivers.addIfAbsent(new DriverInfo(driver, da));
} else {
// This is for compatibility with the original DriverManager
throw new NullPointerException();
}
println("registerDriver: " + driver);
}
Copy the codeWe can see how Java’s SPI mechanism loads Mysql driver classes and registers Mysql driver classes in DriverManager’s registeredDrivers collection.
5.3 Connect to the database using the previously registered Mysql driver class
Now that the Mysql driver class is registered, when will it be used?
To connect to the Mysql database, we need to use the Mysql driver class, right? At this time we returned to the JDBC connection test code JdbcTest class = DriverManager. GetConnection (” JDBC: mysql: / / localhost: 3306 / JDBC “, “root”, “123456”); In this code, take a look at the source of getConnection:
// DriverManager.java
@CallerSensitive
public static Connection getConnection(String url, String user, String password) throws SQLException {
java.util.Properties info = new java.util.Properties();
if(user ! =null) {
info.put("user", user);
}
if(password ! =null) {
info.put("password", password);
}
// Continue to call the getConnection method to connect to the database
return (getConnection(url, info, Reflection.getCallerClass()));
}
Copy the codeFollow up with the getConnection method:
// DriverManager.java
private static Connection getConnection( String url, java.util.Properties info, Class
caller) throws SQLException { ClassLoader callerCL = caller ! =null ? caller.getClassLoader() : null;
synchronized(DriverManager.class) {
// synchronize loading of the correct classloader.
if (callerCL == null) { callerCL = Thread.currentThread().getContextClassLoader(); }}if(url == null) {
throw new SQLException("The url cannot be null"."08001");
}
println("DriverManager.getConnection(\"" + url + "\")");
// Walk through the loaded registeredDrivers attempting to make a connection.
// Remember the first exception that gets raised so we can reraise it.
SQLException reason = null;
// Iterate through the registeredDrivers collection. Note that the previously loaded Mysql driver class instance is registered into this collection
for(DriverInfo aDriver : registeredDrivers) {
// If the caller does not have permission to load the driver then
// skip it.
// Check whether permission exists
if(isDriverAllowed(aDriver.driver, callerCL)) {
try {
println(" trying " + aDriver.driver.getClass().getName());
// Connect to the database using the Mysql driver class
/************* [mainline, focus] *****************/
Connection con = aDriver.driver.connect(url, info);
// Other loaded driver classes such as FabricMySQLDriver are ignored as they are returned directly below as long as they are connected
if(con ! =null) {
// Success!
println("getConnection returning " + aDriver.driver.getClass().getName());
return(con); }}catch (SQLException ex) {
if (reason == null) { reason = ex; }}}else {
println(" skipping: "+ aDriver.getClass().getName()); }}// if we got here nobody could connect.
if(reason ! =null) {
println("getConnection failed: " + reason);
throw reason;
}
println("getConnection: no suitable driver found for "+ url);
throw new SQLException("No suitable driver found for "+ url, "08001");
}
Copy the codeAs you can see, the getConnection method of DriverManager takes the Mysql driver class that was loaded from the registeredDrivers collection to connect to the database.
Well, at this point, the JDBC driver loaded source code is basically analyzed.
Thread context class loader
We have basically analyzed the JDBC driver loading source code, but there is one important point that has not been covered, and that is the thread-context classloader that breaks the parent-delegate model of class loading.
Driver and java.sql.DriverManager are all in the Jdk’s Rt.jar package, meaning that these classes will be loaded by the BootstrapClassLoader; The Mysql driver class is implemented by an external database vendor. When the driver class is imported into the project, it is also in the classpath of the project.
Because of the shortcomings of the parent delegation model of class loading, the parent delegation model has to be broken. Since the classes in the project classpath are loaded by the AppClassLoader, can we “reverse” the launcher class loader to delegate the application class loader to load these external database vendor driver classes? If so, how do we get the launcher class loader to delegate the application class loader to load classes in the classpath?
The answer is yes, we can put the application classloader in the thread, so that the thread defines a new contextClassLoader property, and then at some point put the application classloader in the thread’s contextClassLoader property, If not set, the default is the application class loader. Driver and Java.sql.DriverManager classes are then started, and contextClassLoader is fetched from the current thread to load the JDBC Driver class provided by an external vendor in the classpath. Therefore, the thread-context class loader solves this problem perfectly by breaking the parent-delegate model of the class loading mechanism.
When did you get the thread context class loader when loading the Mysql driver?
The answer is ServiceLoader
loadedDrivers = Serviceloader.load (driver.class); Fetching the thread context classloader from the ServiceLoader load method:
public static <S> ServiceLoader<S> load(Class<S> service) {
ContextClassLoader; contextClassLoader; contextClassLoader; contextClassLoader
ClassLoader cl = Thread.currentThread().getContextClassLoader();
// Pass in the thread context classloader as an argument to load external vendor driver classes in the classpath
return ServiceLoader.load(service, cl);
}
Copy the codeSo, at this point, we understand the role of the thread-context classloader in loading the JDBC driver package. In addition, we should be aware that the vast majority of Java loading involving SPI is done using thread context class loaders, such as JNDI,JCE,JBI, etc.
Extensions: Hot deployment of code breaks the parent-delegate model of class loading, and Tomcat’s class loading mechanism is also worth reading.
Note: If you are not sure about the parent delegate model of class loading, we recommend that you fully understand the parent delegate model and custom classloaders.
7 Extension: Dubbo’s SPI mechanism
As mentioned earlier, the Dubbo framework is full of application of SPI mechanism. It can be said that the application of SPI mechanism is full of extension points. But instead of adopting the default Java SPI mechanism, Dubbo implemented an SPI mechanism of his own.
So why didn’t Dubbo adopt Java’s SPI mechanism?
There are two main reasons:
- Java’s SPI mechanism instantiates all implementations of extension points at once. If there is an extension implementation, initialization is time-consuming, but loading it without using it can be a waste of resources.
- Java’s SPI mechanism does not have support for Ioc and AOP, so Dubbo has used its own SPI mechanism: added support for extension points Ioc and AOP, where one extension point can directly setter and inject other extension points.
For the above reasons, Dubbo has a custom SPI mechanism for loading its own extension points. The mechanism of Dubbo’s SPI is not detailed here. If you are interested, you can go to the official website of Dubbo to see how to extend Dubbo’s SPI. Duboo’s SPI source code analysis article is also available on its website.
8 subtotal
The SPI mechanism of Java is now fully understood.
- The use of Java’s SPI mechanism;
- The principle of Java SPI mechanism;
- Loading principle of JDBC driver;
- The role of thread context classloader in JDBC driver loading;
- The SPI mechanism of Duboo is briefly described.
Due to the author’s limited level, if there are mistakes in the article, please point out, thank you.
Reference:
1, dubbo.apache.org/zh-cn/docs/…
2. Understanding the Java Virtual Machine