1. Background

In a Spring Boot project, redis is used as the cache, so spring-boot-starter-data-redis is used, depending on the following:

<dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-redis</artifactId> < version > 2.0.4. RELEASE < / version > < / dependency >Copy the code

In the test environment, no problems were found in the functional test and pressure test. The reason is that Redis built itself in the test environment and did not set the password. However, after going online, Redis used the cluster of Azure Pass service and set the password.

  1. Redis has high load;
  2. Redis exception, error message:

com.lambdaworks.redis.RedisException: java.lang.IllegalArgumentException: Connection to XXX.XX.XXX.XXX:15000 not allowed. This connection point is not known in the cluster viewjava.lang.IllegalArgumentException: Connection to XXX.XX.XXX.XXX:15000 not allowed. This connection point is not known in the cluster viewConnection to Xxx.xx.xxx.XXX:15000 not allowed. This connection point is not known in the cluster view;

2. Problem analysis + solution

2.1 High load of Redis

2.1.1 Cause of the problem

The original plan was to check whether the code logic problem caused the high load of Redis, so I logged in the Redis server and used the monitor command to observe the frequency of command execution. It was found that Auth password command was executed every time the command was executed, indicating that the connection pool was not properly used and a command was executed to create a connection. This results in high load and inefficient code execution.

2.1.2 solutions

Then, the project using JedisCluster has no such problem, so it is suspected to be the problem of spring-boot-starter-data-redis RedisTemplate. The spring-data-redis driver package was replaced with Lettuce after a certain version. Jedis connection pool is invalid after clustering is enabled. The incorrect configuration is as follows:

# error configuration # Redis configuration spring. Redis.. Cluster nodes = 127.0.0.1:6379 # # # connection timeout time (milliseconds) spring. Redis. Timeout = 60000 Spring. Redis. Password = # XXXXXXX connection pool maximum number of connections (use a negative value indicates no limit) spring. Redis. Jedis. Pool. The Max - active = 8 # # connection pool biggest jam waiting time, If use negative said there is no limit to the spring. Redis. Jedis. Pool. The Max - wait = 1 # # connection pool in the largest free connection spring. Redis. Jedis. Pool. The Max - idle = 8 # minimum idle connections in the connection pool spring.redis.jedis.pool.min-idle=0Copy the code

You need to change the configuration to the correct one. This phenomenon does not occur after the modification. The specific configuration is as follows:

Stand-alone version:

Host =127.0.0.1 spring.redis.port=6379 ### Connection timeout (ms) spring.redis.timeout=60000 Spring. Redis. Password = # XXXXXXX connection pool maximum number of connections (use a negative value indicates no limit) spring. Redis. Jedis. Pool. The Max - active = 8 # # connection pool biggest jam waiting time, If use negative said there is no limit to the spring. Redis. Jedis. Pool. The Max - wait = 1 # # connection pool in the largest free connection spring. Redis. Jedis. Pool. The Max - idle = 8 # minimum idle connections in the connection pool spring.redis.jedis.pool.min-idle=0Copy the code

The cluster version:

# version # Redis cluster configuration spring. Redis.. Cluster nodes = 127.0.0.1:6379 # # # connection timeout time (milliseconds) spring. Redis. Timeout = 60000 Spring. Redis. Password = # XXXXXXX connection pool maximum number of connections (use a negative value indicates no limit) spring. Redis. Lettuce. The pool. The Max - active = 8 # # connection pool biggest jam waiting time, If use negative said there is no limit to the spring. Redis. Lettuce. The pool. The Max - wait = 1 # # connection pool in the largest free connection spring. Redis. Lettuce. The pool. The Max - idle = 8 # minimum idle connections in the connection pool spring.redis.lettuce.pool.min-idle=0Copy the code

Note: To enable cluster edition, you need to add the following additional dependencies

Mons < dependency > < groupId > org.apache.com < / groupId > < artifactId > Commons - pool2 < / artifactId > < version > 2.8.0 < / version > </dependency>Copy the code

2.2 Redis abnormal Connection to XXX.xx.xxx.XXX:15000 not allowed Problem

2.2.1 Cause of the problem

I searched online and found that there is feedback and solution of this problem on github github.com/lettuce-io/… One configuration items, the reason is because the Lettuce is validateClusterNodeMembership default is true cause;

2.2.2 Solutions

Due to the spring boot failure to directly through the configuration file directly modify the configuration, so you need to custom Redis configuration, specific code is as follows: MylettuceConnectionFactory. Java

package com.quison.test.config; import io.lettuce.core.AbstractRedisClient; import io.lettuce.core.cluster.ClusterClientOptions; import io.lettuce.core.cluster.ClusterTopologyRefreshOptions; import io.lettuce.core.cluster.RedisClusterClient; import org.springframework.beans.DirectFieldAccessor; import org.springframework.data.redis.connection.RedisClusterConfiguration; import org.springframework.data.redis.connection.lettuce.LettuceClientConfiguration; import org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory; import org.springframework.data.redis.connection.lettuce.LettucePoolingClientConfiguration; import java.util.concurrent.TimeUnit; public class MyLettuceConnectionFactory extends LettuceConnectionFactory { public MyLettuceConnectionFactory() { } public MyLettuceConnectionFactory(RedisClusterConfiguration redisClusterConfiguration, LettuceClientConfiguration lettuceClientConfiguration) { super(redisClusterConfiguration, lettuceClientConfiguration); } @Override public void afterPropertiesSet() { super.afterPropertiesSet(); DirectFieldAccessor accessor = new DirectFieldAccessor(this); AbstractRedisClient client = (AbstractRedisClient) accessor.getPropertyValue("client"); if(client instanceof RedisClusterClient){ RedisClusterClient clusterClient = (RedisClusterClient) client; ClusterTopologyRefreshOptions topologyRefreshOptions = ClusterTopologyRefreshOptions.builder() .enablePeriodicRefresh(10, TimeUnit.MINUTES) .enableAllAdaptiveRefreshTriggers() .build(); ClusterClientOptions ClusterClientOptions = ClusterClientOptions. Builder () / / note that this configuration item is set to false .validateClusterNodeMembership(false) .topologyRefreshOptions(topologyRefreshOptions) .build(); clusterClient.setOptions(clusterClientOptions); }}}Copy the code

After the connection pool is configured, you need to set the connection pool by yourself. Therefore, the Redis configuration file is changed to redisconfig.java

package com.quison.test.config; import org.apache.commons.pool2.impl.GenericObjectPoolConfig; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Value; import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; import org.springframework.boot.autoconfigure.data.redis.RedisProperties; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.context.annotation.Primary; import org.springframework.data.redis.connection.*; import org.springframework.data.redis.connection.lettuce.DefaultLettucePool; import org.springframework.data.redis.connection.lettuce.LettuceClientConfiguration; import org.springframework.data.redis.connection.lettuce.LettucePoolingClientConfiguration; import org.springframework.data.redis.core.RedisTemplate; import org.springframework.data.redis.serializer.RedisSerializer; import org.springframework.data.redis.serializer.StringRedisSerializer; import java.time.Duration; import java.util.Arrays; import java.util.HashSet; import java.util.Set; @Configuration public class RedisConfig { @Value("${spring.redis.cluster.nodes}") private String clusterNodes; @Value("${spring.redis.password}") private String password; @Value("${spring.redis.lettuce.pool.max-idle}") private Integer maxIdle; @Value("${spring.redis.lettuce.pool.max-active}") private Integer maxActive; @Value("${spring.redis.cluster.max-redirects}") private Integer maxRedirects; @Bean public RedisConnectionFactory myRedisConnectionFactory() { RedisClusterConfiguration redisClusterConfiguration = new RedisClusterConfiguration(); String[] serverArray = clusterNodes.split(","); Set<RedisNode> nodes = new HashSet<RedisNode>(); for (String ipPort : serverArray) { String[] ipAndPort = ipPort.split(":"); nodes.add(new RedisNode(ipAndPort[0].trim(), Integer.valueOf(ipAndPort[1]))); } redisClusterConfiguration.setPassword(RedisPassword.of(password)); redisClusterConfiguration.setClusterNodes(nodes); redisClusterConfiguration.setMaxRedirects(maxRedirects); GenericObjectPoolConfig genericObjectPoolConfig = new GenericObjectPoolConfig(); genericObjectPoolConfig.setMaxIdle(maxIdle); genericObjectPoolConfig.setMinIdle(8); genericObjectPoolConfig.setMaxTotal(maxActive); genericObjectPoolConfig.setMaxWaitMillis(10000); LettuceClientConfiguration clientConfig = LettucePoolingClientConfiguration.builder() .commandTimeout(Duration.ofMillis(10000)) .poolConfig(genericObjectPoolConfig) .build(); return new MyLettuceConnectionFactory(redisClusterConfiguration, clientConfig); } /** * redis template, store the key is a string, the value is Jdk serialized ** @param myRedisConnectionFactory * @return * @description: */ @Bean @ConditionalOnMissingBean(name = "redisTemplate") @Primary public RedisTemplate<? ,? > redisTemplate(RedisConnectionFactory myRedisConnectionFactory) { RedisTemplate<? ,? > redisTemplate = new RedisTemplate<>(); redisTemplate.setConnectionFactory(myRedisConnectionFactory); // Key serialization; However, if the method has a non-string type such as Long, a conversion error will be reported. RedisSerializer<String> redisSerializer = new StringRedisSerializer(); redisTemplate.setKeySerializer(redisSerializer); redisTemplate.setHashKeySerializer(redisSerializer); / / default JdkSerializationRedisSerializer serialization way; StringRedisSerializer stringSerializer = new StringRedisSerializer(); redisTemplate.setKeySerializer(stringSerializer); redisTemplate.setValueSerializer(stringSerializer); redisTemplate.setHashKeySerializer(stringSerializer); redisTemplate.setHashValueSerializer(stringSerializer); return redisTemplate; }}Copy the code

3, summarize

A fall into the pit, a gain in your wisdom, summarized as follows:

  1. The development + test environment should be consistent with the online environment as far as possible, so that problems can be found in advance.
  2. New technologies require a lot of testing before they go into production;