This is the second day of my participation in Gwen Challenge

Question origin

Mall search was implemented based on Solr, but now it is replaced with ElasticSerch, which is better distributed, according to the requirements of the leader. After some research, the data synchronization solution is chosen to use Canal and Canal Adapter for produced by Ali The canal Adapter that you downloaded from Github has been failing since the deployment.

Localization mentality

The phenomenon of description

Download the binary packages for the original Canal and Canal_adapter from Github. The full import of the canal_adapter test is fine, but the incremental import is not possible

Viewing startup Logs

When investigating the Canal Adapter, I found that there was an error reported in the startup log. The original Canala Adapter would continue to start if there was an error when it was started. I was a little idealistic, thinking that there would be no problem if it could be started.

The startup log displays the following error:

Load canal adapter: es7 failed
java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassCastException: com.alibaba.druid.pool.DruidDataSource cannot be cast to com.alibaba.druid.pool.DruidDataSource
Copy the code

Experienced old bird should be able to guess the problem at a glance, I did not do much Java before, so took some detours, the following record positioning steps

Run source code location problem

Download the source of the Canal Adapter, run the debug, and finally locate a catch exception:

DruidDataSource dataSource = DatasourceConfig.DATA_SOURCES.get(config.getDataSourceKey());
Copy the code

A quick note on some of the points excluded:

  • DATA_SOURCES is a HashMap. Check that the Map is not empty
  • [HashMap] [put] [get] [HashMap] [put] [get] [HashMap] [put] [get] [HashMap] [put] [get] [get] [HashMap] [put] [get
  • Since it is doing type conversion error, try not to take out the specific type, but use Object to access the Object, the result is no problem when taking out, but in the end call method in the class error — exclusion

Analyzing dependencies

Druid dependency:tree: MVN Dependency :tree: druID dependency:tree: druID Dependency :tree: DruID Dependency :tree: DruID Dependency :tree: DruID Dependency :tree: DruID Dependency :tree: DruID Dependency :tree

[WARNING] The POM for com. Alibaba: druid: jar: 1.2.6 is invalid, transitive dependencies (if any) will not be available, enable debug logging for more detailsCopy the code

Run the following command to view the details of the error:

mvn -X -U dependency:tree
Copy the code

Details are as follows:

[WARNING] The POM for com. Alibaba: druid: jar: 1.2.6 is invalid, transitive dependencies (if any) will not be available: 2 Problems were encountered while building the effective model for com.alibaba: Druid :1.2.6 [ERROR] 'dependencies.dependency.systemPath' for com.sun:tools:jar must specify an absolute path but is ${project. The basedir} / lib/its - 1.8 - view the jar @ [ERROR] 'dependencies. Dependency. SystemPath' for com. Sun: jconsole: jar Must specify an absolute path but is ${project.basedir}/lib/ openJDK-1.8-jconsole.jar @Copy the code

Druid’s packages are downloaded directly from the web, so I don’t have to dig into the problem. Druid’s packages are compiled directly from the web. Druid’s packages are compiled directly from the web

Root cause: ClassLoader

In the confusion, I googled and saw someone with the same problem. Then I went to Github and looked at the issue. There was this problem, which probably means that the problem was reported in the previous alpha version, but it still hasn’t been solved in the latest release

The root cause of the problem is the use of two class loaders to load the Druid package.

1). When the application starts, the AppClassLoader loads the Druid

2). Es7x-1.1.5-snapshot-jar-with-dependencies. Jar contains druid’s package, so the custom URLClassExtensionLoader comes from client-Adapter. Es7x-1.1.5 – snapshot-jar-with-dependencies. Jar

3). Because the HashMap put() and get() operations operate in different modules, druid loaded by the AppClassLoader is used for PUT () and Drui loaded by the URLClassExtensionLoader is used for get() C) druid cannot be cast; d) Druid cannot be cast

The solution

In pom.xml in the client-Adapter esCore Module, locate the DruID dependency configuration and add the scope

<dependency>
    <groupId>com.alibaba</groupId>
    <artifactId>druid</artifactId>
    <scope>provided</scope>
</dependency>
Copy the code

Setting scope to provided means that the dependency is only used at test and compile time, but not at run time. The end result is that es’s XXXX-with-dependency. Jar does not contain druid