Review the directory

  • The business scenario | synchronous implementation: kill Redis in seconds kill function of practice
  • Design patterns | a word understand design patterns
  • Apache SkyWalking | SkyWalking advanced play
  • Apache SkyWalking | SkyWalking Java plug-in contribution to practice
  • A distributed job scheduling | HOME LINK distributed operation platform (recommended)

background

With the continuous expansion of shell’s business functions, the single application with complex functions has entered the iterative mode of microservice development. Project need job scheduling module is a common requirement, in before monomer system, integrated with quartz to complete the job scheduling module, because single application integration, after a single almost no new work, from the technical level and the overall is relatively stable, but when the monomer used for micro service after break up, a lot of micro service need to integrate all job scheduling module, Some conventional job scheduling implementations have been unable to meet the expansion of microservice projects at the company level, so a lightweight, distributed and unified management job scheduling framework is imperative.

demand

  • Requirements from the company’s perspective: Due to the monomer used to micro services, micro services job scheduling function flowers, job scheduling system in the overall project system belongs to the project infrastructure components, need easy access to the company’s overall monitoring and specification, principle should be entirely, such as task execution appear problem, should be timely report to the police in order to follow up in time, regular daily, Summarize the overall job scheduling of microservice system. The principle of job scheduling should be 100% mastered, and it is not allowed to repair blocked faults due to unclear technical principles on line. When there are more and more access jobs, there should be a load mechanism to monitor the overall system running status.
  • Requirements from the perspective of business system: The job scheduling system should be very easy to access, and access problems should be solved by a special team. The development of business system should not be blocked, and jobs should be flexibly configured and scheduled accurately. The user manual should be strong enough, and the job scheduling task force should continue to support, if necessary, fast-developing business projects that focus on business implementation rather than basic component implementation details and don’t want to get stuck in component usage. Namely: low input, high return.

Technology research

  • The current popular job scheduling project has been investigated on Github, which can be roughly divided into the following forms:
  • 1 deeply optimized and customized Quartz form: Deep understanding of learning the transformation of Quartz source code, Quartz DB to use the migration to use ZK, through the development of back-end management system operation ZK, so as to manage the configuration of the entire job scheduling task, this form of job scheduling, in principle, job scheduling is still completed in the client, the management system is very weak. Such a job scheduling system has an inherent advantage: the failure of the management system will not affect the continued operation of the job. Because client projects integrate almost all of the job scheduling code implementation.

  • 2. Quartz server form: Abstract and encapsulate Quartz as a server. The client is exposed to the hook method of completing tasks through basic client-refence.jar, and the server completes job configuration tasks. In this form of job scheduling framework, the client reduces the amount of job scheduling code and project performance consumption to a certain extent, but brings many distributed scheduling problems, so it is necessary to deeply understand the implementation of the overall scheduling.

  • 3. Self-realization scheduling system. This kind of job scheduling system basically has a lot of functions. Cronexpression. Java implements Cron jobs that resolve tasks at a certain time in the future when the trigger time is reached. Triggering can support client integration or a protocol that triggers client project tasks to complete scheduling. Learning this kind of job scheduling architecture can be thought-provoking, whether in breadth or depth, can enable learners to better understand the job scheduling architecture.

Thinking and Practice

Based on the understanding of the background requirements of the operating system and the technical investigation and study on Github, we have considered the following points in the process of realizing our own job scheduling based on our own characteristics.

  • Scheduling design and implementation: the client exposes tasks, synchronizes tasks with server data through registration and discovery, and assembs the discovered tasks into customized jobs on the server to complete the overall job scheduling function. The server side integrates corporate monitoring and specifications.
    • Overall architecture design
    • Job scheduling principle realization
  • Job scheduling scheme of realization of high availability of a service: the server by zk master for elections, to complete the server itself high availability, in addition the server interface exposed health, here our company has the continuous monitoring of the interface to service health, if cluster, a healthy interface appears unusual, will immediately report to the police to the personnel on duty.
  • Job scheduling server implementation: here we refer to a large number of technical research in the third form of function implementation, through the learning of scheduling bottom implementation, to achieve the task on the server configurable jobs.
  • How to achieve minimal client-side implementation: The job scheduling requirements project exposes task methods to the registry by integrating client-refence.jar and internally implements task triggering methods to complete task scheduling triggering. Here we recognize the annotation method as a task method by adding specific annotations to the method through reflection.
  • Task method registration discovery: as mentioned earlier in the minimalist implementation of the client side, task registration discovery is achieved by placing the identified task methods on the registry and achieving continuous consistency between local exposure tasks and tasks in the registry via heartbeat.
  • Project Authority design: With the client project more and more access to the work platform, definitely need a client access permissions system to complete the project of logical isolation, here we refer to git project personnel management system, the project name is a unique identifier, creating students for this project owner of the project, according to the system, invite other students to join the project, Maintain the project job configuration together.
  • Abnormal alarm realization: define abnormal event types and contents, such as task failure events and abnormal information, and complete real-time alarm of abnormal events through observer mode and project personnel. With the permission member above, send it to the project team member of the task immediately.
  • Dependency, fragmentation task implementation: here we refer to some foreign scientific research projects, combined with the industry more popular design, through the use of ordered acyclic graph (DAG) to achieve the task series dependency, and task level fragmentation.

conclusion

In the process of in-depth study of job scheduling framework, basically all frame analytic Cron job is through the org. Quartz. CronExpression. Java realization of parsing, by parsing the trigger point, after complete the trigger. Job scheduling system in the overall project belongs to is the underlying service component, the basis of the real question is: a lot of business project actually don’t care about the underlying project implementation, as long as the study of low cost, high availability, has a special team to maintain, so doing the design of the project implementation, to focus on the core of user demands. Job scheduling system in generating rarely need to push task in every second, central demand is not high concurrency, so we design the system of index is defined as a second grade quota of trigger delay metrics, there is underlying system must be large, and the study of low cost, to provide a default set of recommended configuration, also should have advanced to meet various business line of conventional custom configuration,