Creating a Task

This article is based on the xxL-job Quick Start Guide. Last time, we briefly introduced the origin and use of XXL-job. This article will introduce some advanced use methods and features.

Above we found that there are a lot of options when creating a new task, now let’s talk more about what they do.

Routing strategy

The routing policy indicates that a task can be completed by multiple actuators. Which one is completed depends on the specified routing policy. This parameter is only meaningful when the actuators are deployed in a cluster.

Quartz can only be loaded randomly

So the first one here, the last one in what order, is to go to 1, 2, 3, 4, the first one refers to 1, and the last one refers to 4.

The operation mode

There are two running modes, BEAN and GLUE

  • BEAN mode: This is the Java class written in the project, and then filled in the JobHandler with the @xxlJob name, is written on the executor side
  • GLUE mode: Supports Java, Shell, Python, PHP, Nodejs, PowerShell, and the code is maintained directly in the dispatch center

I’ve created a GLUE(Java) mode task here, click on the GLUE IDE in actionThis is where we can edit the code we want to executeHowever, there is a hidden security problem. There is no authentication and the method is simple. You only need to add the same token to the application.properties of the scheduling center and the executor.

xxl.job.accessToken=

Blocking handling strategy

The blocking processing policy means that the task execution time is three minutes, but the task execution frequency is set to once every two minutes. In this case, the task execution time is not finished. What happens to the second time?

There are three options, as shown below:

The subtasks

When we write a Job, the tasks are interdependent. Let’s say I have so many things to do, A done B, B done C, C done D.There are two ways to approach this problem. The first way is to write all this logic as one big job, serialized. The second is to use subtasks, which trigger another task at the end of one task.

If you need to trigger another task when the execution of this task ends and succeeds, you can execute the other task as a sub-task of this task. Since each Job has its own unique ID, you only need to fill in the task ID in the sub-task column.

Task timeout

A timeout means that if no result is returned within a specified time, the result is not to be waited for.

Advanced task Usage

Shard task

Let’s say we have a task that needs to process 200,000 pieces of data. The business logic processing of each piece of data takes 0.1 seconds, which is 5.5 hours for a common single thread task. When you want to speed up, your first reaction is to open multi-threaded running, I open 3, almost is more than an hour can be done, your idea is exactly right!

This will be the case where there are three threads working furiously on actuator 0, but is that a good idea? Oh no, I’m tired out here with actuator 0, you’re resting there with actuator 1 and actuator 2, dry and dry and wet and wet and wet and dry, first of all it’s a problem of uneven distribution. Second, when the three threads of executor 0 are working, it will waste its resources, so that the performance of the server will also decline, so this is a bad way.

This is where we come in for our sharding task, and the really good solution is the following, which is a scientific and reasonable solution. Each of the three actuators starts a threadcommonFinish this task! Now, there’s a problem, three machines all running the same piece of code, isn’t that a mess? You run the data once, I run it once, it runs it again.The solution is very simple, one actuator handles 1/3 of the total, we need to do the work of the average divided, I do 1/3, you do 1/3, it does 1/3, so there will be no conflict.

Shard task at run time, the scheduler will send each actuator a different subdivision number, fragmentation of the largest number is the total amount is the same as the actuator, ensure each actuator will perform to the task, such as in the figure above the first actuator get subdivision number 0, the second actuator to shard serial number 1, the third stage get subdivision number 2 actuators. Now it is easy to do, we just need to process the data mod 3, the remainder of the data is dried by actuator 0, the remainder of the data is dried by actuator 1, the remainder of the data is dried by actuator 2.

Our SQL for retrieving data can be written like this:

//Count Total number of fragments, index Number of current fragmentsselect id,name,password from student where status=0 and mod(id,#{count})=#{index} order by id desc limit 1000;
Copy the code

Implement this in your codeWhen creating a task, select Fragmented broadcast and fill in JobHandlerFinally, the amount of data shard does not have to beCompletely equalThe amount of data,The above module is just an example, a train of thought. We can also replace 0, 1, 2 with other conditions to get some data from all the data, such as machine shard number 0 and I look up data from 2018, machine shard number 1 and I look up data from 2019, machine shard number 2 and I look up data from 2020. It’s up to our business to decide exactly how to divide it.

If the nodes are added or decreased, the total number of slices and the maximum number of slices will change in real time.

Command task

For example, you need to periodically restart the database, periodically back up data files (cp tar rm), and periodically clear logs (rm).

The command line is also easy to use, just pass in the executed command as a parameterNow let’s create a new command line taskStart the task, and every 5 seconds, the calculator pops up automatically

Periodic task

A periodic task is one that executes custom methods at the start and end of a task to do whatever you want

conclusion

Guys, this is the end of the two tutorials on XXL-job, and I would like to conclude the last part.

Dispatch center

Responsible for managing scheduling information and issuing scheduling requests according to scheduling configuration, without undertaking business codes. The decoupling of scheduling system and task improves system availability and stability, and the performance of scheduling system is no longer limited by task modules.

The scheduling center supports visual, simple and dynamic management of scheduling information, including task creation, update, deletion, GLUE development and task alarm. All the above operations will take effect in real time, and it also supports monitoring of scheduling results and execution logs and Failover.

The scheduling center puts scheduling requests into an asynchronous scheduling queue. Theoretically, a single scheduling center in the default configuration can support 5000 concurrent tasks and run stably. However, due to network delay, different DB read and write time, and different task scheduling intensity, the maximum number of tasks will fluctuate up and down. To support more tasks, you can increase the number of scheduling threads, reduce the ping delay between the scheduling center and the executor, and improve machine configurations.

xxl.job.triggerpool.fast.max=200
xxl.job.triggerpool.slow.max=100
Copy the code

actuator

Responsible for receiving scheduling requests and executing task logic, task module focuses on the execution of tasks and other operations, development and maintenance more simple and efficient;

Receives execution requests, termination requests, and log requests from the dispatch center.

On the whole, xxL-Job architecture relies less, has powerful functions, is simple but not simple, and is easy to deploy and use. Original is not easy, feel good, please point a thumbs-up!