Life is too short to have a dog
The background,
In daily business, there are more or less such requirements that need to perform a task at a specified time, or perform a task periodically. Tasks like this can generally be boiled down to scheduled tasks. As the saying goes: Where there is demand, there is creation. In order to meet the needs of scheduled tasks, various task scheduling frameworks emerge. The Timer, ScheduledThreadPoolExecutor (what? You read that right, this also works for scheduled tasks), Quartz, etc. However, with the development of distributed, micro-services, the above job scheduling framework is not enough. The main problems are as follows:
- Task running data cannot be obtained, such as task list, task execution status, etc.
- Task parameters cannot be dynamically modified without restarting the application (without using the configuration center).
- The life cycle of a task cannot be performed without restarting the application, such as starting, terminating, restarting, and deleting the task.
- Lack of task failure alarm mechanism
- Not applicable in distributed scenarios
Below, Xianyu makes a simple comparative analysis of the three distributed task scheduling tools commonly used in the market according to his understanding.
Comparison of common distributed scheduling tools
According to the results of research on several major blog platforms and GitHub, there are three commonly used open-source distributed task scheduling frameworks: Elastic Job(Dangdang), Saturn(Vip.com), and XXL-Job (Yelp – Personal).
A simple introduction
Elastic-Job: Elastic-job is an open-source distributed scheduling solution of Dangdang. It consists of two independent sub-projects, elastic-job-Lite and elastic-job-cloud. Elastical-job-lite is positioned as a lightweight, decentralized solution that provides distributed task coordination services in the form of JAR packages. Elastic Job-Cloud uses the self-developed Mesos Framework to provide additional capabilities for resource governance, application distribution, and process isolation. (For now, we’re using Elastic job-Lite.)Saturn: Saturn (Task scheduling system) is an open source distributed task scheduling platform of ViP.com. It replaces the traditional Linux Cron/Spring Batch Job and implements unified configuration, unified monitoring, high availability of tasks, and fragmented concurrent processing. It should be noted that Saturn is a distributed task scheduling framework for secondary development based on elastic-job. Therefore, Saturn has almost all the features of elastic-job.xxl-jobXxl-job is a distributed task scheduling platform opened by Xu Xuili from Dianping in 2015. Initially, XXL-Job was developed as an extension of Quartz, using database locks to ensure that only one node performs the task. In the latest versions, XXL-Job reconstructs the main logic of task scheduling, removes Quartz and selects self-developed RPC module. It is designed to be quick to develop, simple to learn, lightweight, easy to expand, and out of the box.
Functionally, all three open source frameworks meet everyday business needs. It should be noted that the decentralization and service autonomy of elastic-job-Lite are more in line with the distributed concept, while Staurn and XXL-Job retain the concept of dispatch center. Another significant difference is that while the first two use Zookeeper as the service registry, XXL-Job uses the DB approach for task registration and discovery.
Feature comparison
| features | Elastic-Job | xxl-job | Saturn |
|---|---|---|---|
| High availability | By registering and discovering ZooKeeper, you can dynamically add servers | Competitive database lock ensures that only one node performs the task and supports horizontal expansion. You can manually add scheduled tasks, start and pause tasks, and monitor them | With Elastic – the Job |
| Task fragmentation | can | can | can |
| Management interface | There is an operation and maintenance platform, providing quick modification of job Settings, job and server dimension status view, operation job disable, enable, stop and delete life cycle, but the stopped task can not be restarted, no alarm mechanism | Create and edit tasks, edit GLUE code, disable/enable/stop/delete tasks, view scheduling logs, progress monitoring, task dependency, data encryption, email alarm (provides a nail alarm API, can be secondary development), run reports, internationalization | Similar to Elastic-Job, it can restart tasks and provide Executor operation and maintenance functions, such as one-click traffic extraction and one-click dump |
| Ease of development | simple | simple | simple |
Overall, the three frameworks are roughly the same in terms of functionality, but differ in some advanced features. In terms of operations platform, however, elastic-Job is a bit less important than the latter two. For Elastic-Job-Lite, the operations platform is an optional component (currently not used in our project). At the same time, if you want to fully control Job distribution through o&M platform, you need to use elastic-job-cloud. Elastic-job-lite does not have Job distribution.
Usage and community activity
Here’s a direct look at the Star, Issue, contributor parameters on Github.
1. Elastic-Job
Github elasticJob/elasticjob-lite Elastic-Job is a distributed scheduled job framework, based on Quartz and Zookeeper.
2. Saturn
Github address: Saturn
3. xxl-job
Xl-job: A distributed task scheduling framework. (XL-Job)
In terms of popularity and detail on Github, XXL-Job is at the top of the list. Compared to elastic jobs, xxL-jobs have high code maintenance and community activity. In this respect, XXL-jobs are very user-friendly.
Third, summary
The above is a simple analysis and comparison of the three commonly used distributed task scheduling tools. Of course, in addition to these three relatively popular tools, there are also such as TBSchedule(an open source job scheduling framework based on Timer in the early stage of Taobao), light-task-schedule and other job scheduling frameworks, which will not be introduced here. Interested students can search on Github to learn about them.
This article was typeset using MDNICE