preface

Lao Huang met a data cleaning requirement some time ago. In fact, he cleaned yesterday’s data every morning and classified it.

This is a typical timing task processing scenario.

Timed task can be said to be a sharp tool, almost every company can not do without, its application scenarios are not a few, such as:

  1. Generate statistics for the previous day
  2. Clear logs every few days
  3. Deal with expired documents on a regular basis
  4. .

The following are common solutions for scheduled tasks

  1. quartz.net
  2. hangfire
  3. xxl-job
  4. saturn
  5. .

With 1 and 2, there is definitely a learning cost to getting used to, and the downside is that it doesn’t allow developers to focus on the business side of things.

For 3 and 4, these are distributed task scheduling platforms that are well decoupled from services and can trigger the execution of tasks via HTTP interfaces.

For 3 and 4 to be highly available in a production environment, cluster deployment is a must, and it is not easy to deploy such a set of things when resources are tight.

For the above schemes, Huang did not adopt them, but replaced them with CRONJob of K8S.

Why cronJob

The reasons for not choosing the schemes mentioned above have been expressed, which are cost and complexity. Let’s talk about the reasons for choosing cronjob of K8S.

First of all, the cronjob of K8S itself can be regarded as a task scheduling platform. During the scheduling, a POD will be created to execute our tasks.

Second, there are no complex dependencies, just write a simple console program.

Another problem is the cost. The K8S used by Old Yellow Company is serverless, there is no real server resources, only images are delivered, and these scheduled tasks are billed by the time.

1C2G configuration only 0.00006126 yuan per second, assuming that your task execution takes 3 minutes, then this task is only 10 cents.

The choice of configuration comes down to the needs of your business.

With all that said, let’s have a fake example.

A simple example

To prepare for our task, we will write a simple console program.

using System;

internal class Program
{
    private static void Main(string[] args)
    {
        // Write the contents of the scheduled task
        
        Console.WriteLine($"Hello World!  {a}"); }}Copy the code

One important thing to note here is that don’t have console. ReadLine, console. ReadKey, etc., otherwise run won’t work.

Dockerfile is not written, as long as the console program can be packaged as an image, can be run.

Write cronJob configuration.

apiVersion: batch/v1beta1
kind: CronJob
metadata:
  labels:
    etl: diagnosis
  name: xyzxyz
  namespace: prod
spec:
  # Disable concurrent running
  concurrencyPolicy: Forbid
  failedJobsHistoryLimit: 1
  jobTemplate:
    metadata: {}
    spec:
      Specifies the lifetime
      activeDeadlineSeconds: 1200
      # specify 2 retries on failure
      backoffLimit: 2
      completions: 1
      parallelism: 1
      template:
        spec:
          containers:
            - env:
                - name: DOTNET_RUNNING_IN_CONTAINER
                  value: 'true'
                - name: TZ
                  value: Asia/Shanghai
              image: >- xxxxx:5000/xxxxx:version              imagePullPolicy: IfNotPresent
              name: xyzxyz
              ports:
                - containerPort: 80
                  protocol: TCP
              resources:
                # Here only 0.25C 0.5g is used
                requests:
                  cpu: 250m
                  memory: 512Mi
              terminationMessagePath: /dev/termination-log
              terminationMessagePolicy: File
          dnsPolicy: ClusterFirst
          restartPolicy: Never
          schedulerName: default-scheduler
  # cron,
  schedule: 22 4 1/ 1 * ?
  Number of successful jobs displayed in history
  successfulJobsHistoryLimit: 1
Copy the code

In fact, there are a lot of configuration, for the old Huang’s scene, the above configuration is enough, for more configuration, you can refer to the K8S official website.

Execute kubectl apply -f xxx.yml to create a scheduled task, which will be automatically scheduled to execute the corresponding task.

I’m not going to execute it here, but I’m going to take the three timed tasks running on the line for your reference.

Click the details to see the specific execution status.

Write in the last

There are many ways to answer this question, and you can choose the best solution for your company.

Because each solution has its advantages and disadvantages, k8S cronJob also has its disadvantages. The most obvious one is that the cron expression bit 1 is minute instead of second, that is to say, the minimum granularity is only up to minutes. If your application needs to be up to second, it may not be able to support it.