One, foreword
We often contact with all sorts of pooling of technology or concepts, including object pool, connection pool, pool, etc., the biggest benefit of pooling technology for the achievement of object reuse, especially create and use a large object or valuable resources (HTTP connection object, MySQL connection objects), and other aspects when can greatly save the overhead, It is also important to improve the overall performance of the system.
Under concurrent requests, creating/closing MySQL connections for hundreds of Query operations at the same time, or creating a processing thread for each HTTP request, or creating a parsing object for each image or XML parsing without using pooling techniques can be a huge load challenge for the system.
This paper mainly analyzes the implementation scheme of Commons-pool2 pooling technology, hoping to give readers a more comprehensive understanding of the implementation principle of Commons-pool2.
Commons-pool2 pooling technology analysis
More and more frameworks are choosing to use Apache Commons-pool2 for pooled management. For example, Jedis-cluster, commons-pool2 works as follows:
2.1 Core three elements
2.1.1 ObjectPool
Object pool, which manages the life cycle of objects and provides statistics on active and idle objects in the object pool.
2.1.2 PooledObjectFactory
The object factory class is responsible for the creation, initialization, destruction, and verification of the object state. The Commons-pool2 framework itself provides a default abstract implementation, BasePooledObjectFactory, which businesses simply inherit and then warp and create.
2.1.3 PooledObject
The pooled object is a wrapper class that needs to be put into the ObjectPool object. Added some additional information, such as status information, creation time, activation time, etc. Commons-pool2 provides DefaultPooledObject and PoolSoftedObject implementations. PoolSoftedObject inherits from DefaultPooledObject, except that SoftReference is used to implement a SoftReference. You can obtain an object using SoftReference.
2.2 Logical Analysis of object Pools
2.2.1 Description of Object Pool Interfaces
1) When using Commons-pool2, an application obtains or releases an object based on the object pool. The core interfaces of the object pool are as follows:
/** * add object instances */ to the object pool
void addObject(a) throws Exception, IllegalStateException,
UnsupportedOperationException;
/** * gets the object */ from the object pool
T borrowObject(a) throws Exception, NoSuchElementException,
IllegalStateException;
/** * invalid invalid object */
void invalidateObject(T obj) throws Exception;
/** * release object to object pool */
void returnObject(T obj) throws Exception;
Copy the codeIn addition to the interface itself, object pooling supports setting the maximum number of objects, retention time, and so on. The core parameters of the object pool include maxTotal, maxIdle, minIdle, maxWaitMillis, testOnBorrow, etc.
2.2.2 Object Creation decoupling
Object factory is the core link used to generate objects in commons-Pool2 framework. Business parties need to implement the corresponding object factory implementation classes in the process of use. Through the factory mode, the object pool is decoupled from the details of object generation and implementation process. This decouples the object pool itself from the object generation logic.
We can further verify our thinking with the code:
public GenericObjectPool(final PooledObjectFactory<T> factory) {
this(factory, new GenericObjectPoolConfig<T>());
}
public GenericObjectPool(final PooledObjectFactory<T> factory,
final GenericObjectPoolConfig<T> config) {
super(config, ONAME_BASE, config.getJmxNamePrefix());
if (factory == null) {
jmxUnregister(); // tidy up
throw new IllegalArgumentException("factory may not be null");
}
this.factory = factory;
idleObjects = new LinkedBlockingDeque<>(config.getFairness());
setConfig(config);
}
public GenericObjectPool(final PooledObjectFactory<T> factory,
final GenericObjectPoolConfig<T> config, final AbandonedConfig abandonedConfig) {
this(factory, config);
setAbandonedConfig(abandonedConfig);
}
Copy the codeAs you can see, the object pool constructors rely on the object construction factory PooledObjectFactory, which generates objects based on the parameters defined in the object pool and the object construction factory.
/** * Adds objects to the object pool. This is usually used during preloading
@Override
public void addObject(a) throws Exception {
assertOpen();
if (factory == null) {
throw new IllegalStateException(
"Cannot add objects without a factory.");
}
final PooledObject<T> p = create();
addIdleObject(p);
}
Copy the codeThe create() method is based on the objects generated by the object factory, working down the code to validate the logic;
final PooledObject<T> p;
try {
p = factory.makeObject();
if(getTestOnCreate() && ! factory.validateObject(p)) { createCount.decrementAndGet();return null; }}catch (final Throwable e) {
createCount.decrementAndGet();
throw e;
} finally {
synchronized(makeObjectCountLock) { makeObjectCount--; makeObjectCountLock.notifyAll(); }}Copy the codeThe factory.makeObject() operation is confirmed here, confirming the above assumption that the corresponding object is generated based on the object factory.
To better enable the use of objects in object pools and to track their state, the commons-pool2 framework uses the concept of PooledObject PooledObject, which is itself a generic class and provides methods for getObject() to get the actual object.
2.2.3 Object Pool source code analysis
After the above analysis, we know that the object pool carries the management of the life cycle of the object, including the control of the number of objects in the whole object pool and other logic. Next, we analyze how to achieve it through GenericObjectPool source code.
LinkedBlockingDeque provides operations to insert and remove elements at the end of the queue and at the head of the queue. All operations are performed to add a reentrant lock to the queue. Operations such as await and notify are triggered when an operation is performed on a queue element.
/ * * * * the first node Invariant: (first = = null && last = = null) | | * (first. Prev = = null && first. The item! = null) */
private transient Node<E> first; // @GuardedBy("lock")
/ * * * the last node * Invariant: (first = = null && last = = null) | | * (last. Next = = null & & last. Item! = null) */
private transient Node<E> last; // @GuardedBy("lock")
/** The current queue length */
private transient int count; // @GuardedBy("lock")
/** Maximum queue capacity */
private final int capacity;
/ * * * / master lock
private final InterruptibleReentrantLock lock;
/** Whether the queue is an empty state lock */
private final Condition notEmpty;
/** Whether the queue is full is locked */
private final Condition notFull;
Copy the codeThe core point of the queue is:
1. All moving in, moving out, initializing construction elements in the queue are locked based on the primary lock.
2. Queue offer and pull support setting timeout parameters, mainly through the two Condition to coordinate operations. For example, if the offer operation fails, wait based on the notFull state object.
public boolean offerFirst(final E e, final long timeout, final TimeUnit unit)
throws InterruptedException {
Objects.requireNonNull(e, "e");
long nanos = unit.toNanos(timeout);
lock.lockInterruptibly();
try {
while(! linkFirst(e)) {if (nanos <= 0) {
return false;
}
nanos = notFull.awaitNanos(nanos);
}
return true;
} finally{ lock.unlock(); }}Copy the codeFor example, if the pull operation fails, notEmpty waits.
public E takeFirst(a) throws InterruptedException {
lock.lock();
try {
E x;
while ( (x = unlinkFirst()) == null) {
notEmpty.await();
}
return x;
} finally{ lock.unlock(); }}Copy the codeOtherwise, when the operation is successful, the wake up operation is carried out, as shown below:
private boolean linkLast(final E e) {
// assert lock.isHeldByCurrentThread();
if (count >= capacity) {
return false;
}
final Node<E> l = last;
final Node<E> x = new Node<>(e, l, null);
last = x;
if (first == null) {
first = x;
} else {
l.next = x;
}
++count;
notEmpty.signal();
return true;
}
Copy the code2.3 Core business processes
2.3.1 Status of the Pooled Object Changed
All states are defined in the PooledObjectState class, some of which are temporarily unused and won’t be covered here.
2.3.2 browObject process of object Pool
The first step is to determine, based on the configuration, whether to call the removeAbandoned method for tag removal.
The second step, try to get or create an object, the source process is as follows:
// The pollFirst method is a non-blocking method
p = idleObjects.pollFirst();
if (p == null) {
p = create();
if(p ! =null) {
create = true; }}if (blockWhenExhausted) {
if (p == null) {
if (borrowMaxWaitMillis < 0) {
//2. If the maximum blocking wait time is not set, wait indefinitely
p = idleObjects.takeFirst();
} else {
//3, set the maximum wait time, block the specified wait timep = idleObjects.pollFirst(borrowMaxWaitMillis, TimeUnit.MILLISECONDS); }}}Copy the codeThe schematic diagram is as follows:
3. Call ALLOCATE to change the state to ALLOCATED.
Step 4. Call the factory activateObject to initialize the object. If an error occurs, call the Destroy method to destroy the object, as in step 6 in the source code.
Fifth, call TestFactory’s validateObject for an availability analysis based on the TestOnBorrow configuration, and call destroy if the object is not available. The source code for steps 3-7 is shown below:
// Modify the object status
if(! p.allocate()) { p =null;
}
if(p ! =null) {
try {
// Initialize the object
factory.activateObject(p);
} catch (final Exception e) {
try {
destroy(p, DestroyMode.NORMAL);
} catch (final Exception e1) {
}
}
if(p ! =null && getTestOnBorrow()) {
boolean validate = false;
Throwable validationThrowable = null;
try {
// Verify the availability status of the object
validate = factory.validateObject(p);
} catch (final Throwable t) {
PoolUtils.checkRethrow(t);
validationThrowable = t;
}
// The object is not available. If authentication fails, destroy
if(! validate) {try {
destroy(p, DestroyMode.NORMAL);
destroyedByBorrowValidationCount.incrementAndGet();
} catch (final Exception e) {
// Ignore - validation failure is more important}}}}Copy the code2.3.3 Process logic for returnObject in the object pool
The first step is to change the state to RETURNING by calling the markReturningState method.
The second step, based on testOnReturn configuration, calls PooledObjectFactory’s validateObject method for availability checking. If the check fails, call Destroy to consume the object, then make sure that IDLE is called to ensure that idle state objects are available in the pool, and if not, call the create method to create a new object.
The third step is to call PooledObjectFactory’s passivateObject method for de-initialization.
4. Call deallocate to change the state to IDLE.
Fifth, check whether the maximum number of free objects has been exceeded, if so, destroy the current object.
Step 6. Place the object at the beginning or end of the queue according to LIFO (LIFO) configuration.
2.4 Expand and think
2.4.1 Another implementation of LinkedBlockingDeque
Commons-pool2 uses dual-ended queues and Condition in Java to manage objects in queues and coordinate the operations of different threads on acquiring and releasing objects. Is there any other solution that can achieve similar effects? The answer is yes.
In essence, we use two queues to store the idle queue and the current active object respectively, and then use a unified object lock. The same goal can be achieved. The general idea is as follows:
1. Two one-way queues are used to store idle and active objects respectively. Synchronization and coordination between queues can be completed through wait and Notify of object locks.
public class PoolState {
protected final List<PooledObject> idleObjects = new ArrayList<>();
protected final List<PooledObject> activeObjects = new ArrayList<>();
/ /...
}
Copy the codeWhen fetching objects, LIFO or FIFO is used to fetch objects from idleObjects, and then add the objects to activeObjects after obtaining the objects successfully and the object status is legal. If fetching objects requires waiting, The PoolState object lock should enter the wait state through the wait operation.
3, in the release of the object, first from the activeObjects set activeObjects to delete the element, after the completion of the deletion, add the object to the idle object set idleObjects, it should be noted that in the release of the object process also need to verify the state of the object. Objects should be destroyed when their state is invalid and should not be added to idleObjects. After the release is successful, PoolState uses notify or notifyAll to wake up the fetch operation in wait.
To ensure thread safety for active and idle queues, lock objects and release objects, as in commons2-pool.
2.4.2 Self-Protection mechanism of object Pools
When we get an object using Commons-pool2, we block the queue waiting to get an element (or create a new object), but if the application has an exception and no returnObject or invalidObject is called, In that case, the objects in the object pool may rise all the time, and when borrowObject is called after reaching the set upper limit, there will be the situation that the objects cannot be obtained due to waiting or waiting timeout.
Commons-pool2 provides two self-protection mechanisms to avoid the problems analyzed above:
2.4.2.1 Threshold based detection
When obtaining objects from the object pool, the system verifies the proportion of the number of active objects and idle objects in the current object pool. When the number of idle objects is very small and the number of active objects is very large, the system triggers the recycling of idle objects. The verification rules are as follows: If there are less than 2 idle objects in the current object pool or the number of active objects is greater than the maximum number of objects -3, leak cleaning is started when borrow objects. Through AbandonedConfig. SetRemoveAbandonedOnBorrow to true to open it.
// Determine whether to call the removeAbandoned method for tag removal based on the configuration
final AbandonedConfig ac = this.abandonedConfig;
if(ac ! =null && ac.getRemoveAbandonedOnBorrow() && (getNumIdle() < 2) && (getNumActive() > getMaxTotal() - 3) ) {
removeAbandoned(ac);
}
Copy the code2.4.2.2 Asynchronous scheduling thread detection
AbandonedConfig. SetRemoveAbandonedOnMaintenance is set to true, at the time of maintenance operation will be leaking objects clear, By setting the setTimeBetweenEvictionRunsMillis to set the maintenance task execution time interval.
The startEvictor method is called at the end of the constructor’s internal logic. The purpose of this method is to start the collector to monitor the collection of free objects after the object pool is constructed. StartEvictor is defined in the BaseGenericObjectPool (Abstract) class of GenericObjectPool’s parent class. Let’s look at the source of this method.
The following setup parameters are executed in the constructor;
public final void setTimeBetweenEvictionRunsMillis(
final long timeBetweenEvictionRunsMillis) {
this.timeBetweenEvictionRunsMillis = timeBetweenEvictionRunsMillis;
startEvictor(timeBetweenEvictionRunsMillis);
}
Copy the codeIf and only if timeBetweenEvictionRunsMillis parameters has been set to open regular cleaning tasks.
final void startEvictor(final long delay) {
synchronized (evictionLock) {
EvictionTimer.cancel(evictor, evictorShutdownTimeoutMillis, TimeUnit.MILLISECONDS);
evictor = null;
evictionIterator = null;
// If delay<=0, the scheduled clearing task is not enabled
if (delay > 0) {
evictor = newEvictor(); EvictionTimer.schedule(evictor, delay, delay); }}}Copy the codeFollowing the code, we can see that the implementation logic of the cleanup method set up in the scheduler is actually defined in the object pool, which is implemented by GenericObjectPool or GenericKeyedObjectPool. Next, we can explore how object pooling performs object collection.
A) Core parameters:
MinEvictableIdleTimeMillis: free object specified maximum retention time, and more than this time will be recycled. If this parameter is not configured, the collection will not expire.
SoftMinEvictableIdleTimeMillis: a millisecond value, which is used to specify in the free objects more than minIdle Settings, and a free object more than the free time to just can be recycled.
MinIdle: Indicates the minimum number of space objects to be reserved in the object pool.
B) Recycle logic
EvictionPolicy is the object collection policy interface, and EvictionPolicy is the object collection policy interface. It is expected that the object pool collection will be associated with the above parameters and interface EvictionPolicy.
boolean evict;
try {
evict = evictionPolicy.evict(evictionConfig, underTest,
idleObjects.size());
} catch (final Throwable t) {
// Slightly convoluted as SwallowedExceptionListener
// uses Exception rather than Throwable
PoolUtils.checkRethrow(t);
swallowException(new Exception(t));
// Don't evict on error conditions
evict = false;
}
if (evict) {
// Call destroy directly if it can be recycled
destroy(underTest);
destroyedByEvictorCount.incrementAndGet();
}
Copy the codeIn order to improve the efficiency of recycling, when the state of the object judged by the recycling strategy is not EVICT, further state judgment and processing will be carried out, and the specific logic is as follows:
1. Try to activate the object. If the activation fails, the object is no longer alive and is destroyed by calling Destroy.
2. If the object is activated successfully, the validateObject method is used to obtain the status of the verification object. If the verification fails, the object is unavailable and needs to be destroyed.
boolean active = false;
try {
// Call activateObject to activate the idle object.
// There may be some resource exploitation in this step.
factory.activateObject(underTest);
active = true;
} catch (final Exception e) {
// If an exception occurs during activation, the idle object is disconnected.
// Call the destroy method to destroy underTest
destroy(underTest);
destroyedByEvictorCount.incrementAndGet();
}
if (active) {
// Verify the validity with validateObject
if(! factory.validateObject(underTest)) {// If the verification fails, the object is unavailable
destroy(underTest);
destroyedByEvictorCount.incrementAndGet();
} else {
try {
/* * Since the checksum also activates idle objects and allocates additional resources, resources created in the activateObject are released via passivateObject. * /
factory.passivateObject(underTest);
} catch (final Exception e) {
// If passivateObject fails, the idle object underTest is unavailabledestroy(underTest); destroyedByEvictorCount.incrementAndGet(); }}}Copy the codeThird, write at the end
Connection pooling can bring some convenience to application developers. In the introduction, we analyze the benefits and necessity of using pooling technology, but we can also see that the Commons-pool2 framework locks the creation and acquisition of objects, which can affect application performance to a certain extent in concurrent scenarios. Secondly, the number of objects in the object pool of pooled objects also needs to be set reasonably, otherwise it is difficult to achieve the real purpose of using object pool, which brings us certain challenges.
Huang Xiaoqun, Vivo Internet Server Team