Still writing task scheduling codes by hand? Try this visual distributed scheduling framework!

SpringBoot e-commerce project mall (40K + STAR) address: github.com/macrozheng/…

Abstract

Task scheduling scenarios are often used in microservice systems. For example, periodically synchronize data every day, periodically generate service reports, and periodically clear logs. Today I recommend a distributed scheduling framework to help you easily complete the task scheduling work!

PowerJob profile

PowerJob is a new generation of distributed scheduling and computing framework, which enables you to easily complete task scheduling and distributed computing of complex tasks.

Main features:

• Easy to use: Provides a front-end Web interface, allowing developers to visually manage scheduling tasks and view task running status and logs.
• Improved timing policies: Supports CRON expression, fixed frequency, fixed delay, and API timing policies.
• Multiple execution modes: Supports single machine, broadcast, Map, and MapReduce execution modes.
• Dependency thin: minimum dependence only on relational databases (MySQL, etc.), extended dependence to MongoDB (for storing large online logs).

Why is there a dispatch center

We usually use QuartZ or Spring Task to implement scheduled tasks in applications, but in microservices architecture, it is not appropriate for many applications to be riddled with this kind of Task scheduling code. A reasonable solution is that the execution methods of tasks exist in the application, and we have a scheduling center that is responsible for scheduling these methods. We only need to configure tasks in the scheduling center. PowerJob is just such a distributed scheduling framework.

Installation preparation

PowerJob’s scheduling center (PowerJob-Server) needs to use MySQL to store data and MongoDB to store logs, so we install and start these two services first.

• Start MySQL service in Docker container;

docker run -p 3306:3306 --name mysql \
-v /mydata/mysql/log:/var/log/mysql \ -v /mydata/mysql/data:/var/lib/mysql \ -v /mydata/mysql/conf:/etc/mysql \ -e MYSQL_ROOT_PASSWORD=root \ -d Mysql: 5.7Copy the code

• Create a database for PowerJob in MySQLpowerjob-product;

CREATE DATABASE IF NOT EXISTS `powerjob-product` DEFAULT CHARSET utf8mb4
Copy the code

• Start the MongoDB service in the Docker container.

Docker run - p - 27017-27017 the name mongo \ - v/mydata/mongo/db: / data/db \ - d mongo: 4.2.5Copy the code

Installing the Dispatch Center

Installing the dispatch center for PowerJob in Docker environment is very simple and can be done in minutes.

• Download mirrorpowerjob-serverDocker image:

docker pull tjqq/powerjob-server:latest
Copy the code

• Run in a Docker containerpowerjob-serverServices:

docker run -p 7700:7700 -p 10086:10086 --name powerjob-server \
--link mysql:db \
--link mongo:mongo \
-e TZ="Asia/Shanghai" \
-e JVMOPTIONS="" \
-e PARAMS="--spring.profiles.active=product --spring.datasource.core.jdbc-url=jdbc:mysql://db:3306/powerjob-product? useUnicode=true&characterEncoding=UTF-8 --spring.datasource.core.username=root --spring.datasource.core.password=root --spring.data.mongodb.uri=mongodb://mongo:27017/powerjob-product" \
-v ~/docker/powerjob-server:/mydata/powerjob/powerjob-server \
-v ~/.m2:/mydata/powerjob/.m2 \
-d tjqq/powerjob-server:latest
Copy the code

• It can be accessed after running successfullypowerjob-serverThe Linux firewall needs to be open7700and10086Two ports, access address:http://192.168.3.101:7700/

Initializes the actuator in the application

After installing the dispatch center, we need to initialize the PowerJob executor (PowerJob-worker) in the SpringBoot application.

• First of all inpom.xmladdpowerjob-workerIs dependent on:

<dependency>
    <groupId>com.github.kfcfans</groupId>
    <artifactId>powerjob-worker-spring-boot-starter</artifactId>
    <version>3.2.3</version>
</dependency>
Copy the code

• After theapplication.ymlAdded to the configuration filepowerjob-workerNote Related configurationspowerjob.worker.app-nameThis configuration;

powerjob:
  worker:
    akka-port: 27777 # akka working port
    app-name: mall-tiny-powerjob Access application name, used for group isolation
    server-address: 192.1683.101.: 7700 Address of the dispatch server
    store-strategy: disk # Persistence
Copy the code

• Add a stand-alone processorStandaloneProcessor, just inheritBasicProcessorInterface and implementationprocessMethod can;

package com.macro.mall.tiny.job;

@Slf4j
@Component
public class StandaloneProcessor implements BasicProcessor {

    @Override
    public ProcessResult process(TaskContext context){
        //OmsLogger can directly report logs to powerJob-server
        OmsLogger omsLogger = context.getOmsLogger();
        omsLogger.info("StandaloneProcessor start process,context is {}.", context);
        log.info("jobParams is {}", context.getJobParams());
        return new ProcessResult(true."Process success!"); }}Copy the code

• After the image is packaged and uploaded, run the SpringBoot application service in the Docker container. Ensure that the time zone is configured in accordance with the scheduling center.

docker run -p 8080:8080 --name mall-tiny-powerjob \
--link mysql:db \
-v /etc/localtime:/etc/localtime \
-v /mydata/app/mall-tiny-powerjob/logs:/var/logs \
-e TZ="Asia/Shanghai"\ d mall - tiny/mall - tiny - powerjob: 1.0 the SNAPSHOTCopy the code

Task configuration and execution

With the actuator and the scheduling center, we only need to configure the task in the scheduling center to realize the task scheduling.

• First we need to register the application (integration actuator) in the dispatch center, the application name isapplication.ymlIn thepowerjob.worker.app-nameProperty, which is used heremall-tiny-powerjob:123456;

• Then we can see a machine information on the home page;

• After theTask managementAdd a task to the function, here we useCRONsetEvery 20 secondsExecute the processing method in the executor;

• Click on the task listrunStart to perform tasks;

• Click on the task listMore -> Run recordsYou can view task run logs.

• Click on theThe logYou can view the logs reported in the processor.jobParamsIs the parameter we set when we created the task earlier;

• Click on thedetailsYou can view the result of the triggered task, which is that we areProcessResultInformation returned from the.

The resources

Official documentation: github.com/KFCFans/Pow…

Project source code address

Github.com/macrozheng/…

In this paper, making github.com/macrozheng/… Already included, welcome everyone Star!