Introduction⌗

Elastic Crontab System is a distributed scheduled task management system developed in Golang, using ETCD as the core component for service governance. It utilizes ETCD’s key prefix monitoring mechanism to distribute pipelines to Worker nodes.

Architecture⌗

Architecture

Concepts⌗

Pipeline⌗

A Pipeline is a collection of Tasks. You can define a name, description, tasks triggered on success or failure, and scheduling time for a pipeline. You can associate existing tasks within the pipeline. This way, the pipeline can be assigned to specific Worker nodes for scheduling. Only when all tasks in the pipeline are executed successfully is the entire pipeline considered complete. If any task fails in the process, the pipeline’s scheduling result is considered a failure.

Task⌗

A Task is the smallest executable unit in a pipeline and cannot be scheduled independently. Currently, the following task types are supported:

  • Shell tasks
  • Mail notification tasks
  • Hook tasks
  • Http tasks

Each task can be bound to multiple pipelines, which means the same task doesn’t need to be defined multiple times on different nodes, greatly improving management efficiency. Each task records execution logs when scheduled, allowing users to view the results of tasks running on specific nodes for a given pipeline.

Installation⌗

ETCD⌗

For performance considerations, ETCD version 3.3.12 is used. If you only have a single-machine environment, you can deploy a single-node service by running the etcd single command directly.

$ docker run -d \
    --name etcd \
    -p 2379:2379 \
    -p 2380:2380 \
    --volume=/private/var/local/etcd/data:/data \
    gcr.io/etcd-development/etcd:v3.3.12 \
    /usr/local/bin/etcd \
    --name etcd \
    --data-dir /data \
    --listen-client-urls http://0.0.0.0:2379 \
    --advertise-client-urls http://0.0.0.0:2379 \
    --listen-peer-urls http://0.0.0.0:2380 \
    --initial-advertise-peer-urls http://0.0.0.0:2380 \
    --initial-cluster etcd=http://0.0.0.0:2380 \
    --initial-cluster-token betterde \
    --initial-cluster-state new

MySQL⌗

In the development environment, MySQL 8.0.11 is used, so at least MySQL version 7.* or above is required.

$ docker run -d \
    --name mysql \
    -p 3306:3306 \
    -e MYSQL_ROOT_PASSWORD=your-secret-pw mysql:8.0.11 \
    --character-set-server=utf8mb4 \
    --collation-server=utf8mb4_unicode_ci

ECTS⌗

$ git clone [email protected]:betterde/ects.git
$ cd ects && docker build -t ects:latest .
docker run --rm -p 9701:9701 ects:latest ects init

The project provides Docker Compose deployment method

docker-compose up -d

For more information, refer to the official documentation on Service Configuration

Management⌗

For detailed documentation on service management, please refer to Task Management

Contribution⌗

The project is hosted on Github. If you encounter any issues while using it, you can raise questions here or contribute your code. Your attention will help us go further!

I hope this is helpful, Happy hacking…