Read the harvest

✔️1. Learn about the periodic task framework of distributed applications

✔️2. Learn how to set up and use the periodic task framework of XXL-job

Commonly used distributed task scheduling system

  • xxl-job: Distributed task scheduling platform is a lightweight distributed task scheduling framework released by Xu Xueli, an employee of Dianping, in 2015. Its core design goal is rapid development, simple learning, lightweight and easy to expand.
  • Quartz: Java de facto scheduled task standard. But Quartz focuses on timed tasks rather than data, and there is no process that can be customized for data processing. Although Quartz can implement high availability of database-based jobs, it lacks distributed parallel scheduling capabilities
  • TBSchedule: Ali’s early open source distributed task scheduling system. The code is slightly older and uses timers instead of thread pools to perform task scheduling. Timers are known to be defective in handling exceptions. In addition, the TBSchedule job type is relatively simple and can only be a mode of data acquisition/processing. There is also a serious lack of documentation
  • Elastic - job (E - job): Elastic distributed task scheduling system developed by Dangdang has rich and powerful functions. Zookeeper is used to realize distributed coordination, high availability of tasks and sharding. The current version is 2.15, and it can support cloud development
  • Saturn: is a distributed scheduling platform for scheduled tasks independently developed by VipSHOP. Developed based on Dangdang’s Elastice-Job version 1, it can be well deployed on Docker containers.

  • Thing in common:
    • E-job and X-Job have a wide user base and complete technical documents, which meet the basic function requirements of scheduled tasks.
  • The difference between
    • X-job focuses on easy service implementation, easy management, easy learning cost, and rich failure policies and routing policies. It is recommended to use it when the user base is relatively small and the number of servers is within a certain range
    • E-job focuses on data, adding flexible capacity expansion and data fragmentation to maximize the utilization of distributed server resources. However, the learning cost is high. You are advised to use this command when a large amount of data is required and a large number of servers are deployed

xxl-job

Design idea:

  • The scheduling behavior is abstracted into a common platform called “scheduling center”, which does not undertake the business logic itself and is responsible for initiating scheduling requests.
  • The tasks are abstracted into scattered JobHandler and managed by the executor. The executor is responsible for receiving scheduling requests and executing the service logic in the corresponding JobHandler.
  • Therefore, “scheduling” and “task” can be decoupled from each other to improve the overall stability and scalability of the system.
  • Version 2.3.0 is used in this article

1. DownloadThe source code

  • Github download address
  • Gitee download address

2. The initializationThe database

  • Run the tables_xxl_job. SQL file to initialize
  • The scheduling center supports cluster deployment. In cluster deployment, all nodes must be connected to the same mysql instance.
  • If mysql is the primary server, nodes in the scheduling center must forcibly switch to the primary database.
    • Xxl_job_lock: lock table for scheduling tasks;
    • Xxl_job_group: maintains information about task actuators in the actuator information table.
    • Xxl_job_info: used to save extended information about xxL-job scheduling tasks, such as task group, task name, machine address, actuator, execution input parameter, and alarm email.
    • Xxl_job_log: used to save historical information about xxL-job task scheduling, such as scheduling results, execution results, scheduling input parameters, scheduling machines, and actuators.
    • Xxl_job_log_report: scheduling log report: a report that stores xxL-job task scheduling logs, which is used on the report function page of the scheduling center.
    • Xxl_job_logglue: Task GLUE log: Saves the GLUE update history and supports the GLUE version tracing function.
    • Xxl_job_registry: the executor registry, which maintains the online executor and scheduling center machine address information;
    • Xxl_job_user: indicates the system user table.

3. Configure the deploymentDispatch center

  • Scheduling center project: xxl-job-admin
  • Functions: Centrally manages tasks scheduled on the task scheduling platform, is responsible for triggering scheduling execution, and provides a task management platform.

3.1 Main Configuration Modifications:

  • Modifying a data source
  • Modify alarm Mailbox
  • Dispatch center communication TOKEN
### xxl-job, datasource
spring.datasource.url=jdbc:mysql://localhost:3306/xxl_job? useUnicode=true&characterEncoding=UTF-8&autoReconnect=true&serverTimezone=Asia/Shanghai
spring.datasource.username=root
spring.datasource.password=root
spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver

### xxl-job, email
spring.mail.host=smtp.qq.com
spring.mail.port=25
spring.mail.username=[email protected]
spring.mail.from=[email protected]
spring.mail.password=Your mail token
spring.mail.properties.mail.smtp.auth=true
spring.mail.properties.mail.smtp.starttls.enable=true
spring.mail.properties.mail.smtp.starttls.required=true
spring.mail.properties.mail.smtp.socketFactory.class=javax.net.ssl.SSLSocketFactory
### Dispatch center communication TOKEN [optional] : non-space enabled;
### xxl-job, access token
xxl.job.accessToken=ljwtoken
Copy the code

3.2. Package and Deploy the Dispatch Center

  • mvn clean package -Dmaven.test.skip=true

3.3. Access the Scheduling center

  • Dispatching Center access address:http://localhost:8080/xxl-job-admin (The address actuator configuration file xxl.job.admin.addresses will be used as callback addresses)
  • The default login account is admin/123456. The following figure shows the login page.
  • Reference for cluster deployment in the scheduling center

4. Configure the deploymentActuator item

  • The actuator project is as follows: xxl-job-executor-sample-springboot (Multiple versions of the actuator are available. The Springboot version is used as an example.It can be used directly or referenced to transform existing projects into actuators)
  • Role: Responsible for receiving and executing the dispatching of “dispatching center”; Actuators can be deployed directly or integrated into existing business projects.
  • Principle:
    • The executor is actually an embedded Server with the default port 9999 (configuration item: xxl.job.executor.port).
    • At project startup, the executor identifies “bean-mode tasks” in the Spring container with “@jobHandler” and manages them with the annotated value attribute as key.
    • When the executor receives a scheduling request from the scheduling center, if the task type is Bean-mode, it matches the Bean-mode task in the Spring container and invoks its Execute method to execute the task logic. If the task type is “GLUE mode”, GLUE code is loaded, Java objects are instantiated, dependent Spring services are injected (note: Spring services injected in GLUE code must exist in the Spring container with the “executor” project), and the execute method is called to execute the task logic.

4.1 the maven rely on

  • The executor project needs to import maven dependencies for “xxl-job-core” with the same version as the registry version
<dependency>
    <groupId>com.xuxueli</groupId>
    <artifactId>xxl-job-core</artifactId>
    <version>${project.parent.version}</version>
</dependency>
Copy the code

4.2 Modifying the Configuration File

  • xxl-job-executor-sample-springboot/src/main/resources/application.properties
  • Change the deployment and address of the dispatch center
  • Need to modify or customize
    • XXL. Job. Admin. Addresses the address
    • Xxl.job.executor. Appname User-defined name, which must correspond to background configuration
    • Xxl.job.executor. IP The IP address of the current computer, or the computer IP address of the deployed project
    • XXL. Job. Executor. Port port
    • Xxl.job. AccessToken The customized token must be the same as that configured in the admin module
### Dispatch center deployment and ADDRESS [Optional] : If multiple addresses exist in the dispatch center cluster deployment, separate them with commas (,). The executor will use this address for "executor heartbeat registration" and "task result callback". If it is null, auto registration is disabled.
xxl.job.admin.addresses=http://127.0.0.1:8080/xxl-job-admin
### XXL -job, access token User-defined token. The value must be the same as that configured in the admin module
xxl.job.accessToken=ljwtoken
### xxl-job Executor AppName User-defined name, which must correspond to background configuration
xxl.job.executor.appname=xxl-job-executor-sample
### xxl-job executor registry-address: default use address to registry , otherwise use ip:port if address is null
xxl.job.executor.address=
### xxl-job executor server-info
xxl.job.executor.ip=127.0.0.1
xxl.job.executor.port=9999
Copy the code

4.3 Configuring the Actuator Component To configure the actuator component, refer to the following address for the configuration file: /xxl-job/xxl-job-executor-samples/xxl-job-executor-sample-springboot/src/main/java/com/xxl/job/executor/core/config/XxlJ obConfig.java

@Bean
public XxlJobSpringExecutor xxlJobExecutor(a) {
    logger.info(">>>>>>>>>>> xxl-job config init.");
    XxlJobSpringExecutor xxlJobSpringExecutor = new XxlJobSpringExecutor();
    xxlJobSpringExecutor.setAdminAddresses(adminAddresses);
    xxlJobSpringExecutor.setAppname(appname);
    xxlJobSpringExecutor.setAddress(address);
    xxlJobSpringExecutor.setIp(ip);
    xxlJobSpringExecutor.setPort(port);
    xxlJobSpringExecutor.setAccessToken(accessToken);
    xxlJobSpringExecutor.setLogPath(logPath);
    xxlJobSpringExecutor.setLogRetentionDays(logRetentionDays);

    return xxlJobSpringExecutor;
}
Copy the code

4.4 Deploying the Executor Project

  • If the above configuration is correct, compile and deploy the executor project, we use the Springboot project, package and run xxl-job-executor-sample-springboot
  • mvn clean package -Dmaven.test.skip=true
  • Cluster deployment reference for executor project

5. GLUE pattern (Java) Develop a task

  • GLUE pattern (Java) principle: Tasks are maintained in the dispatch center in source code; The task of this pattern is actually a piece of Java class code inherited from IJobHandler and maintained as “Groovy” source code, which runs in the executor project, You can use @Resource/@Autowire to inject other services in the executor (make sure the service and class references in the Glue code are present in the “executor” project) and then call the execute method on that object to execute the task logic.

  • The GLUE pattern (Java) simply writes code in the interface, not in the executor project

5.1 Creating a Task

  • Log in to the tuning center and click the “Add” button as shown below to create a new example task. Then, refer to the task parameters in the screenshot below and click Save.

5.2 “GLUE Mode (Java)” Task Development

  • Please click the “GLUE IDE” button on the right of the task to enter the “GLUE Editor Development interface”, as shown in the picture below. Tasks in “GLUE mode (Java)” run mode already initialize the sample task code by default, printing Hello World.

5.3 Triggering Execution

  • Click The “Execute Once” button on the right of a task to manually trigger the execution of a task (usually, you can configure the Cron expression to trigger the task scheduling).

5.4 Viewing Logs:

  • Click The Scheduling Log button on the right of a task to view task logs on the task log page.

  • On the task log page, you can view the historical scheduling records of a task and the scheduling information, execution parameters, and execution information of each scheduled task. Running Tasks Click The Execution Log button on the right to access the log console to view real-time execution logs.

6. Develop a task in BEAN mode

  • Principle of BEAN mode: Tasks are maintained on the executor side in JobHandler mode. You need to incorporate “JobHandler”. The task class needs to be annotated with “@jobHandler (value=” name “)” because the “executor” will recognize tasks in the Spring container based on this annotation. Task classes must inherit IJobHandler. The task logic is developed in the Execute method. When the executor receives a scheduling request from the scheduling center, it invokes the Execute method of IJobHandler to execute the task logic.

6.1 class form

6.1.1 the pros and cons

  • Bean-mode tasks that support class-based development, with one Java class for each task.
    • Advantages:
      • Does not limit the project environment, good compatibility. Even frameless projects, such as those launched directly with the main method, can be supported
    • Disadvantages:
      • Each task requires one Java class, resulting in a waste of classes;
      • Automatic scan tasks are not supported and injected into the actuator container. Manual injection is required.

In the 6.1.2 Actuator project, develop the Job class

  • 1, develop an inherited from “com. XXL. Job. Core. Handler. IJobHandler” JobHandler, implement the task.
  • 2. Manually inject it into the actuator container in the following way.
    • : XxlJobExecutor registJobHandler (” demoJobHandler “, new demoJobHandler ());

Note: V2.2.0 removes the old class annotation @jobHandler, and recommends using the method annotation @xxlJob for task development. (If you want to retain the usage of class annotation JobHandler, you can refer to the old version of logic custom development);

public class MyJobHandler extends IJobHandler {
    @Override
    public void execute(a) throws Exception {
        XxlJobHelper.log("11==============MyJobHandler=================");
        System.out.println("22==============MyJobHandler================="); }}Copy the code

Injection myJobHandler:

@SpringBootApplication
public class XxlJobExecutorApplication {

    public static void main(String[] args) {
        SpringApplication.run(XxlJobExecutorApplication.class, args);

        XxlJobExecutor.registJobHandler("myJobHandler".newMyJobHandler()); }}Copy the code

6.1.3 Creating a scheduling task in the Scheduling Center

  • JobHandler Enter the value of @jobHandler

  • You can run a test to check whether the logs are created successfully

6.2 Method Form

6.2.1 the pros and cons

  • For method-based tasks, the underlying JobHandler agent is generated, and as with class-based tasks, the task exists in the executor task container as JobHandler.
    • Advantages:
      • You only need to develop one method for each task and add the @xxlJob annotation to make it easier and faster.
      • Supports automatic scanning tasks and injection into the actuator container.
    • Disadvantages: Requires a Spring container environment

In the 6.2.2 actuator project, develop the Job method

    1. Task development: In the Spring Bean instance, develop the Job method
    1. Annotation configuration: @xxlJob (value=” custom jobHandler name “, init =” jobHandler initialization method “, destroy =” jobHandler destroy method “) The annotation value corresponds to the value of the JobHandler property for the new task in the dispatch center.
    1. Execution logs: Run xxljobhelper. log to print execution logs
    1. Task result: The default task result is Succeeded and you do not need to set it. If there are any demands, such as setting up task results for failure, can pass “XxlJobHelper. HandleFail/handleSuccess” setting the task results
@Configuration
public class MyXxlJobConfig {

    @XxlJob("myXxlJobConfig")
    public boolean demoJobHandler(a) throws Exception {
        XxlJobHelper.log("========myXxlJobConfig============");
        System.out.println("========myXxlJobConfig============");
        / / ReturnT has no effect
        //return new ReturnT(200, "hahahahah");
        return XxlJobHelper.handleSuccess("myXxlJobConfig hello world");
    }


    @XxlJob("myfaile")
    public boolean myfaile(a) throws Exception {
        XxlJobHelper.log("========myfaile============");
        System.out.println("========myfaile============");
        return XxlJobHelper.handleFail("myfaile hello world");
    }
    
 
    /** * 5, life cycle task example: task initialization and destruction, support custom related logic; Init is executed only once on the first schedule. Destroy executes */ for each dispatch
    @XxlJob(value = "demoJobHandler2", init = "init", destroy = "destroy")
    public void demoJobHandler2(a) throws Exception {
        System.out.println("========demoJobHandler2============");
        XxlJobHelper.log("XXL-JOB, Hello World.");
    }

    public void init(a) {
        System.out.println("========init============");
        logger.info("init");
    }

    public void destroy(a) {
        System.out.println("========destory============");
        logger.info("destory"); }}Copy the code

6.2.3 Creating a scheduling task in the Scheduling Center

  • Change JobHandler to @xxlJob (“myfaile”) and create a new job

  • You can run a test to check whether the logs are created successfully
  • Myfaile returns xxljobHelper. handleFail(“myfaile hello world”); Error, but the task was scheduled successfully

Note:

  • Detailed description of configuration properties

Description of task execution results

The system determines the execution result based on the following criteria

  • Mission success: XxlJobHelper handleSuccess ()
  • Task execution failed: xxljobhelper.handlefail ()
  • Note: Mission dispatch room success, here is only the judgment of business return results

Terminates a running task

This applies only to executing tasks. On the task log interface, click the “Terminate Task” button on the right, and the task termination request will be sent to the corresponding executor of the task, which will terminate the task and empty the entire task execution queue.

The task terminates using an “interrupt” thread of execution, which raises an “InterruptedException” exception. So if JobHandler catches the exception internally and digests it, the task termination function will not be available.

Therefore, if the above task termination is not available, special handling (thrown upward) for the “InterruptedException” exception should be required in JobHandler with the following logic:

try{
    // do something
} catch (Exception e) {
    if (e instanceof InterruptedException) {
        throw e;
    }
    logger.warn("{}", e);
}
Copy the code

Also, when a child thread is started in JobHandler, the child thread must not catch “InterruptedException” and should actively throw it up.

The “destroy()” method corresponding to JobHandler is executed when the job terminates, which handles some of the resource reclamation logic.

Task timeout control

  • You can set the task timeout period. If the task times out, the system interrupts the task.
  • Note that similar to the task termination mechanism (see section “Stopping a running task”), a task is interrupted by “interrupt”, so the business code needs to override “InterruptedException” to make the function unavailable.

The last

  • 👍🏻 : have harvest, praise encouragement!
  • ❤️ : Collect articles, easy to look back!
  • 💬 : Comment exchange, mutual progress!