DevOps 之使用 Consul 管理配置

DevOps 之使用 Consul 管理配置

之前我们都是自己手动 SSH 到远端服务器,然后为项目添加配置文件,于是有些没有些开发者为了省事,将配置的敏感信息也提交到 Git 仓库中,从而导致服务器信息泄露。

Consul 服务

之前我写过一片《管理服务配置》,在这篇文章中,主要讲解了 Consul 搭配 confd 实现服务配置的自动更新。那么在这里就不再过多赘述如何搭建 Consul 的服务了。

创建配置文件模板

建议将项目的配置模板与项目一起放在 Git 仓库中,这样便于管理和部署。比如我们的 Laravel 项目,在项目根目录创建 .env.tpl 文件,内容如下:

.env.tpl
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
APP_NAME={{ key "project/itdb/production/APP_NAME" }}
APP_ENV={{ key "project/itdb/production/APP_ENV" }}
APP_KEY={{ key "project/itdb/production/APP_KEY" }}
APP_DEBUG={{ key "project/itdb/production/APP_DEBUG" }}
APP_URL={{ key "project/itdb/production/APP_URL" }}

LOG_CHANNEL={{ key "project/itdb/production/LOG_CHANNEL" }}

DB_CONNECTION={{ key "project/itdb/production/DB_CONNECTION" }}
DB_HOST={{ key "project/itdb/production/DB_HOST" }}
DB_PORT={{ key "project/itdb/production/DB_PORT" }}
DB_DATABASE={{ key "project/itdb/production/DB_DATABASE" }}
DB_USERNAME={{ key "project/itdb/production/DB_USERNAME" }}
DB_PASSWORD={{ key "project/itdb/production/DB_PASSWORD" }}

BROADCAST_DRIVER={{ key "project/itdb/production/BROADCAST_DRIVER" }}
CACHE_DRIVER={{ key "project/itdb/production/CACHE_DRIVER" }}
QUEUE_CONNECTION={{ key "project/itdb/production/QUEUE_CONNECTION" }}
SESSION_DRIVER={{ key "project/itdb/production/SESSION_DRIVER" }}
SESSION_LIFETIME={{ key "project/itdb/production/SESSION_LIFETIME" }}

REDIS_HOST={{ key "project/itdb/production/REDIS_HOST" }}
REDIS_PASSWORD={{ key "project/itdb/production/REDIS_PASSWORD" }}
REDIS_PORT={{ key "project/itdb/production/REDIS_PORT" }}

MAIL_DRIVER={{ key "project/itdb/production/MAIL_DRIVER" }}
MAIL_HOST={{ key "project/itdb/production/MAIL_HOST" }}
MAIL_PORT={{ key "project/itdb/production/MAIL_PORT" }}
MAIL_USERNAME={{ key "project/itdb/production/MAIL_USERNAME" }}
MAIL_PASSWORD={{ key "project/itdb/production/MAIL_PASSWORD" }}
MAIL_ENCRYPTION={{ key "project/itdb/production/MAIL_ENCRYPTION" }}

如果觉得麻烦你也可以使用如下方式生成:

.env.tpl
1
2
{{ range ls "configs/crm" }}
{{ .Key }}={{ .Value }}{{ end }}

但是这样的话,最终输出的配置文件中配置项的顺序完全是按照字母排序的。

key 后面指定的是你在 Consul 中添加的 KEY 的完整路径

Consul Template

Consul Template 是由 HashiCorp 用 Golang 实现的将 KEY/VALUE 按照模板转换为配置文件的一个开源工具。官方下载链接:https://releases.hashicorp.com/consul-template

下载完成后我们需要将其部署到需要生成配置文件的远程服务器上。

简单部署

1
2
3
consul-template \
-consul-addr=http://consul.example.com:8500 \
-template /path/to/.env.tpl:/path/to/target/.env:'php /path/to/project/artisan config:cache'

请将上面的 /path/to/*/ 替换成你的实际路径

这样 Consul Template 将会一直运行,从而达到监听 Consul 中 KEY/VALUE 的变化,进而修改本地配置文件。但是这种方式不是可取的。

使用 Systemd 管理

创建 /etc/consul-template/config.hcl 配置文件,内容如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
consul {
address = "http://consul.example.com:8500"
retry {
# This enabled retries. Retries are enabled by default, so this is
# redundant.
enabled = true

# This specifies the number of attempts to make before giving up. Each
# attempt adds the exponential backoff sleep time. Setting this to
# zero will implement an unlimited number of retries.
attempts = 12

# This is the base amount of time to sleep between retry attempts. Each
# retry sleeps for an exponent of 2 longer than this base. For 5 retries,
# the sleep times would be: 250ms, 500ms, 1s, 2s, then 4s.
backoff = "250ms"

# This is the maximum amount of time to sleep between retry attempts.
# When max_backoff is set to zero, there is no upper limit to the
# exponential sleep between retry attempts.
# If max_backoff is set to 10s and backoff is set to 1s, sleep times
# would be: 1s, 2s, 4s, 8s, 10s, 10s, ...
max_backoff = "1m"
}
}

# This is the signal to listen for to trigger a reload event. The default
# value is shown below. Setting this value to the empty string will cause CT
# to not listen for any reload signals.
reload_signal = "SIGHUP"

# This is the signal to listen for to trigger a graceful stop. The default
# value is shown below. Setting this value to the empty string will cause CT
# to not listen for any graceful stop signals.
kill_signal = "SIGINT"

# This is the maximum interval to allow "stale" data. By default, only the
# Consul leader will respond to queries; any requests to a follower will
# forward to the leader. In large clusters with many requests, this is not as
# scalable, so this option allows any follower to respond to a query, so long
# as the last-replicated data is within these bounds. Higher values result in
# less cluster load, but are more likely to have outdated data.
max_stale = "10m"

# This is the log level. If you find a bug in Consul Template, please enable
# debug logs so we can help identify the issue. This is also available as a
# command line flag.
log_level = "warn"

# This is the path to store a PID file which will contain the process ID of the
# Consul Template process. This is useful if you plan to send custom signals
# to the process.
pid_file = "/var/run/consul-template.pid"

# This is the quiescence timers; it defines the minimum and maximum amount of
# time to wait for the cluster to reach a consistent state before rendering a
# template. This is useful to enable in systems that have a lot of flapping,
# because it will reduce the the number of times a template is rendered.
wait {
min = "5s"
max = "10s"
}

# This block defines the configuration for a template. Unlike other blocks,
# this block may be specified multiple times to configure multiple templates.
# It is also possible to configure templates via the CLI directly.
template {
# This is the source file on disk to use as the input template. This is often
# called the "Consul Template template". This option is required if not using
# the `contents` option.
source = "/path/on/disk/to/.env.tpl"

# This is the destination path on disk where the source template will render.
# If the parent directories do not exist, Consul Template will attempt to
# create them, unless create_dest_dirs is false.
destination = "/path/on/disk/where/template/will/.env"

# This options tells Consul Template to create the parent directories of the
# destination path if they do not exist. The default value is true.
create_dest_dirs = true

# This is the optional command to run when the template is rendered. The
# command will only run if the resulting template changes. The command must
# return within 30s (configurable), and it must have a successful exit code.
# Consul Template is not a replacement for a process monitor or init system.
command = "php /path/to/project/artisan config:cache"

# This is the maximum amount of time to wait for the optional command to
# return. Default is 30s.
command_timeout = "60s"

# Exit with an error when accessing a struct or map field/key that does not
# exist. The default behavior will print "<no value>" when accessing a field
# that does not exist. It is highly recommended you set this to "true" when
# retrieving secrets from Vault.
error_on_missing_key = false

# This is the permission to render the file. If this option is left
# unspecified, Consul Template will attempt to match the permissions of the
# file that already exists at the destination path. If no file exists at that
# path, the permissions are 0644.
perms = 0600

# This option backs up the previously rendered template at the destination
# path before writing a new one. It keeps exactly one backup. This option is
# useful for preventing accidental changes to the data without having a
# rollback strategy.
backup = true

# These are the delimiters to use in the template. The default is "{{" and
# "}}", but for some templates, it may be easier to use a different delimiter
# that does not conflict with the output file itself.
left_delimiter = "{{"
right_delimiter = "}}"

# This is the `minimum(:maximum)` to wait before rendering a new template to
# disk and triggering a command, separated by a colon (`:`). If the optional
# maximum value is omitted, it is assumed to be 4x the required minimum value.
# This is a numeric time with a unit suffix ("5s"). There is no default value.
# The wait value for a template takes precedence over any globally-configured
# wait.
wait {
min = "2s"
max = "10s"
}
}

如果你在一台服务器上需要部署多个项目,让 Consul Template 监听多个项目的配置信息,那么可以添加多个 template 配置块。更多详细的参数可以参考这里: https://github.com/hashicorp/consul-template#configuration-file-format。

创建 /etc/systemd/system/consul-template.service 文件,内容如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
[Unit]
Description=Consul Template Service
Documentation=https://github.com/hashicorp/consul-template
After=syslog.target network.target
ConditionFileIsExecutable=/usr/lib/consul-template/consul-template

[Service]
Type=simple
Restart=on-failure
StartLimitBurst=10
StartLimitInterval=5
PIDFile=/var/run/consul-template.pid
ExecStart=/usr/lib/consul-template/consul-template --config=/etc/consul-template/config.hcl
ExecReload=/bin/kill -s SIGHUP $MAINPID
ExecStop=/bin/kill -s SIGINT $MAINPID

[Install]
WantedBy=multi-user.target

1
2
3
4
sudo chmod 754 /etc/systemd/system/consul-template.service
sudo systemctl daemon-reload
sudo systemctl enable consul-template.service
sudo systemctl start consul-template.service

这样就完成了 Systemd 对 Consul Tempalate 的进程管理了。

评论