The problem

I had a business that needed to record the number of calls, so I wrote the following code without thinking

public Entry getEntry(a) {... repository.updateCount(id);return entry;
}
Copy the code

Since this query method is called so frequently that it updates the database so frequently, I thought I could cache the data now and periodically update it to the database to reduce database write IO, and I wrote the following code

private final Cache<Integer, AtomicInteger> countCache = CacheBuilder.newBuilder()
    .concurrencyLevel(1)
    .expireAfterAccess(Duration.ofSeconds(10))
    .removalListener((RemovalListener<? super Integer, ? super AtomicInteger>) notification -> {
        Integer id = notification.getKey();
         AtomicInteger count = notification.getValue();

        if(id ! =null&& count ! =null) {
            repository.updateCount(id, count.get());
        }
    })
    .build();

public Entry getEntry(a) {...try {
        AtomicInteger count = countCache.get(entryId, AtomicInteger::new);
        count.incrementAndGet();
    } catch (ExecutionException e) {
        e.printStackTrace();
    }
    return entry;
}
Copy the code

It was perfect. I first created a cache and set it to expire after 10 seconds. When it expired, the number of calls would be updated to the database.

UpdateCount (id, count.get())); This line creates a deadlock, and there’s nothing you can do about it

So I used my careful logical thinking ability, combined with the actual situation, came to the following conclusion

  1. removalListenerNo single thread execution – A single thread will queue up to perform the update and there will be no lock contention
  2. removalListenerNot on a regular basis. If it was on a regular basis, myupdateCountIt won’t take 10 seconds
  3. removalListenerIs checked and executed when the cache is placed or retrieved

Based on the above judgment, the problem is that getEntry() is called in something else that is taking too long for some reason

After reviewing the code, we found that an interface called getEntry() in the transaction and then called the external service, which took too long to respond, causing the transaction to take too long

Here are a few solutions I can think of

  1. updateCountMethods are queued in a single thread, which requires a lot of code
  2. removalListenerEnable independent things so that external things are not affected

I chose plan 2

The cache build becomes the following code

@Autowired
private PlatformTransactionManager transactionManager;

private final Cache<Integer, AtomicInteger> countCache = CacheBuilder.newBuilder()
    .concurrencyLevel(1)
    .expireAfterAccess(Duration.ofSeconds(10))
    .removalListener((RemovalListener<? super Integer, ? super AtomicInteger>) notification -> {
        Integer id = notification.getKey();
        AtomicInteger count = notification.getValue();

        if(id ! =null&& count ! =null) {
            final DefaultTransactionDefinition definition = new DefaultTransactionDefinition(DefaultTransactionDefinition.PROPAGATION_REQUIRES_NEW);
            final TransactionStatus status = transactionManager.getTransaction(definition);
            repository.updateCount(id, count.get());
            transactionManager.commit(status);
        }
    })
    .build();
Copy the code

The problem was solved for the time being.