Making: github.com/jayknoxqu/f…

Overview of connection pools

Frequent and close connections, can greatly reduce the performance of the system, and the connection pool will be at the time of initialization will create a certain number of connections, every visit to just get connection from the connection pool and put it back again after using connection pool, and is not directly close the connection, so that the program to reuse the same connection without the need to build and close connection with each visit, This improves system performance.

Commons-pool2 introduction

2.1 Introduction of POOL2

<! Commons-pool2 -->
<dependency>
    <groupId>org.apache.commons</groupId>
    <artifactId>commons-pool2</artifactId>
    <version>2.5.0</version>
</dependency>

<! -- Introduce FTPClient as pooled object -->
<dependency>
    <groupId>commons-net</groupId>
    <artifactId>commons-net</artifactId>
    <version>3.6</version>
</dependency>
Copy the code

2.2 Composition of POOL2

PooledObject PooledObjectFactory ObjectPool

Corresponding to:

FTPClient(pooled object) FTPClientFactory(object factory) FTPClientPool

Diagram:

Three. Connection pooling

3.1 configuration FtpClient

We already have the pooled object (FtpClient), we just need to add the configuration

@ConfigurationProperties(ignoreUnknownFields = false, prefix = "ftp.client")
public class FtpClientProperties {
    / / FTP address
    private String host;
    / / the port number
    private Integer port = 21;
    // Login user
    private String username;
    // Login password
    private String password;
    // Passive mode
    private boolean passiveMode = false;
    / / code
    private String encoding = "UTF-8";
    // Connection timeout time (seconds)
    private Integer connectTimeout;
    // Buffer size
    private Integer bufferSize = 1024;
    // Transfer file type
    private Integer transferFileType;
}
Copy the code

Application.properties is configured to:

Host =127.0.0.1 ftp.client.port=22 ftp.client.username=root ftp.client.password=root ftp.client.encoding= UTF-8  ftp.client.passiveMode=false
ftp.client.connectTimeout=30000
Copy the code

3.2 create FtpClientFactory

​

There are two kinds of factories in Commons-pool2: PooledObjectFactory and KeyedPooledObjectFactory, which we use.

public interface PooledObjectFactory<T> {
	// Create an object
    PooledObject<T> makeObject(a);
	// Activate the object
    void activateObject(PooledObject<T> obj);
    // Passivate the object
    void passivateObject(PooledObject<T> obj);
    // Validate objects
    boolean validateObject(PooledObject<T> obj);
    // Destroy the object
    void destroyObject(PooledObject<T> obj);
}
Copy the code

​

To create FtpClientFactory, you simply inherit the BasePooledObjectFactory abstract class, which implements PooledObjectFactory

public class FtpClientFactory extends BasePooledObjectFactory<FTPClient> {

    private FtpClientProperties config;

    public FtpClientFactory(FtpClientProperties config) {
        this.config = config;
    }

    /** * Create an FtpClient object */
    @Override
    public FTPClient create(a) {
        FTPClient ftpClient = new FTPClient();
        ftpClient.setControlEncoding(config.getEncoding());
        ftpClient.setConnectTimeout(config.getConnectTimeout());
        try {

            ftpClient.connect(config.getHost(), config.getPort());
            int replyCode = ftpClient.getReplyCode();
            if(! FTPReply.isPositiveCompletion(replyCode)) { ftpClient.disconnect(); log.warn("FTPServer refused connection,replyCode:{}", replyCode);
                return null;
            }

            if(! ftpClient.login(config.getUsername(), config.getPassword())) { log.warn("ftpClient login failed... username is {}; password: {}", config.getUsername(), config.getPassword());
            }

            ftpClient.setBufferSize(config.getBufferSize());
            ftpClient.setFileType(config.getTransferFileType());
            if(config.isPassiveMode()) { ftpClient.enterLocalPassiveMode(); }}catch (IOException e) {
            log.error("create ftp connection failed...", e);
        }
        return ftpClient;
    }

    /** * encapsulates the object into the pool with PooledObject */
    @Override
    public PooledObject<FTPClient> wrap(FTPClient ftpClient) {
        return new DefaultPooledObject<>(ftpClient);
    }

    /** * Destroys the FtpClient object */
    @Override
    public void destroyObject(PooledObject<FTPClient> ftpPooled) {
        if (ftpPooled == null) {
            return;
        }

        FTPClient ftpClient = ftpPooled.getObject();

        try {
            if(ftpClient.isConnected()) { ftpClient.logout(); }}catch (IOException io) {
            log.error("ftp client logout failed... {}", io);
        } finally {
            try {
                ftpClient.disconnect();
            } catch (IOException io) {
                log.error("close ftp client failed... {}", io); }}}/** * Verify the FtpClient object */
    @Override
    public boolean validateObject(PooledObject<FTPClient> ftpPooled) {
        try {
            FTPClient ftpClient = ftpPooled.getObject();
            return ftpClient.sendNoOp();
        } catch (IOException e) {
            log.error("Failed to validate client: {}", e);
        }
        return false; }}Copy the code

3.3 implementation FtpClientPool

In Commons-pool2, three object pools are preset that can be used directly: GenericObjectPool, GenericKeyedObjectPool, and SoftReferenceObjectPool

In the columns:

GenericObjectPool<FTPClient> ftpClientPool = new GenericObjectPool<>(new FtpClientFactory());
Copy the code

We can also implement a connection pool ourselves:

public interface ObjectPool<T> extends Closeable {
    // Get an object from the pool
    T borrowObject(a);
	Return an object to the pool
    void returnObject(T obj);
    // Discard an invalid object
    void invalidateObject(T obj); 
    // Add objects to the pool
    void addObject(a);
	// Clear the object pool
    void clear(a);
    // Close the object pool
    void close(a);
}
Copy the code

ObjectPool is implemented by inheriting BaseObjectPool

public class FtpClientPool extends BaseObjectPool<FTPClient> {

    private static final int DEFAULT_POOL_SIZE = 8;

    private final BlockingQueue<FTPClient> ftpBlockingQueue;
    private final FtpClientFactory ftpClientFactory;


    /** * To initialize the connection pool, inject a factory to provide the FTPClient instance **@paramFtpClientFactory FTP factory *@throws Exception
     */
    public FtpClientPool(FtpClientFactory ftpClientFactory) throws Exception {
        this(DEFAULT_POOL_SIZE, ftpClientFactory);
    }

    public FtpClientPool(int poolSize, FtpClientFactory factory) throws Exception {
        this.ftpClientFactory = factory;
        ftpBlockingQueue = new ArrayBlockingQueue<>(poolSize);
        initPool(poolSize);
    }

    /** * To initialize the connection pool, inject a factory to provide the FTPClient instance **@paramMaxPoolSize Maximum number of connections *@throws Exception
     */
    private void initPool(int maxPoolSize) throws Exception {
        for (int i = 0; i < maxPoolSize; i++) {
            // Add objects to the pooladdObject(); }}/** * get the object */ from the connection pool
    @Override
    public FTPClient borrowObject(a) throws Exception {
        FTPClient client = ftpBlockingQueue.take();
        if (ObjectUtils.isEmpty(client)) {
            client = ftpClientFactory.create();
            // Add the connection pool
            returnObject(client);
            // Verify that the object is valid
        } else if(! ftpClientFactory.validateObject(ftpClientFactory.wrap(client))) {// Process invalid objects
            invalidateObject(client);
            // Create a new object
            client = ftpClientFactory.create();
            // Add the new object to the connection pool
            returnObject(client);
        }
        return client;
    }

    /** * return the object to the connection pool */
    @Override
    public void returnObject(FTPClient client) {
        try {
            if(client ! =null && !ftpBlockingQueue.offer(client, 3, TimeUnit.SECONDS)) { ftpClientFactory.destroyObject(ftpClientFactory.wrap(client)); }}catch (InterruptedException e) {
            log.error("return ftp client interrupted ... {}", e); }}/** * remove invalid object */
    @Override
    public void invalidateObject(FTPClient client) {
        try {
            client.changeWorkingDirectory("/");
        } catch (IOException e) {
            e.printStackTrace();
        } finally{ ftpBlockingQueue.remove(client); }}/** * add a new link */
    @Override
    public void addObject(a) throws Exception {
        // Insert objects into the queue
        ftpBlockingQueue.offer(ftpClientFactory.create(), 3, TimeUnit.SECONDS);
    }

    /** * close connection pool */
    @Override
    public void close(a) {
        try {
            while(ftpBlockingQueue.iterator().hasNext()) { FTPClient client = ftpBlockingQueue.take(); ftpClientFactory.destroyObject(ftpClientFactory.wrap(client)); }}catch (Exception e) {
            log.error("close ftp client ftpBlockingQueue failed... {}", e); }}}Copy the code

It is not advisable to implement connection pooling yourself, which incurs additional maintenance costs…

Iv. Code Address:

Making: github.com/jayknoxqu/f…

V. References:

FTPClient connection pool implementation: yq.aliyun.com/articles/59…

Apache Commons – pool2 (finishing) : www.jianshu.com/p/b0189e01d…

The officer scheme column: commons.apache.org/proper/comm…