Issues Encountered When Setting Up an ETCD Cluster⌗

According to the official ETCD documentation, cluster discovery is broadly divided into two types. One is static discovery, which requires specifying fixed cluster node IPs or hosts before creating the cluster, but this approach is not feasible in cloud-native environments.

So I continued to research the dynamic discovery approach, which requires an ETCD cluster or a single node to store discovery tokens and cluster size. At this point, you might think, like I did, that this is a chicken-and-egg problem. Don’t worry, CoreOS Inc. provides a public Discovery service: https://discovery.etcd.io/. This service is also open-sourced on GitHub at: https://github.com/coreos/discovery.etcd.io.

However, CoreOS Inc.’s project is no longer maintained. In the ETCD project, there is https://github.com/etcd-io/discoveryserver, which provides methods to build your own Docker image.

Preparation⌗

The Discovery Server needs an existing ETCD cluster or single node to store discovery tokens, so we first need to start a single-node ETCD container. Here I’m using the bitnami/etcd:3.5.1 image.

Create Network⌗

docker create network services

Start ETCD Container⌗

docker run -d --name etcd \
  --network services \
  -p 2379:2379 \
  -p 2380:2380 \
  --env ALLOW_NONE_AUTHENTICATION=yes \
  --env ETCD_ADVERTISE_CLIENT_URLS=http://etcd:2379 \
  bitnami/etcd:3.5.1

Start Discovery Server⌗

docker run -d --name discovery \
  --network services \
  -p 8087:8087 \
  -e DISC_ADDR=:8087 \
  -e DISC_HOST=http://localhost:8087 \
  -e DISC_ETCD=http://etcd:2379 \
  betterde/discovery:latest

Configuration Explanation⌗

  • DISC_ADDR: The address and port the service listens on after starting. If IP is not specified, it can be :8087
  • DISC_HOST: Defines the URL the service handles. I understand this to be similar to Nginx’s server_name.
  • DISC_ETCD: Defines the URL of the ETCD service

I’m using an image I built based on the https://github.com/etcd-io/discoveryserver project. Of course, you can also use betterde/discovery directly, which is completely public.

Obtaining a Discovery Token⌗

curl -X PUT localhost:8087/new?size=3

# You'll get a token with a discovery URL as shown below
http://localhost:8087/5c689d7b17d78fd0bd5de1791fa58734

Here, size=3 sets the number of cluster nodes. If not set, the default is 3.

Now let’s see what happened in ETCD:

ETCDCTL_API=3 etcdctl --endpoints http://localhost:2379 get / --prefix

# Below is what the Discovery Server created in ETCD after I requested a discovery token

/discovery.etcd.io/registry/001/k//5c689d7b17d78fd0bd5de1791fa58734/

/discovery.etcd.io/registry/002/k//5c689d7b17d78fd0bd5de1791fa58734/_config/

/discovery.etcd.io/registry/003/k//5c689d7b17d78fd0bd5de1791fa58734/_config/size
5
/discovery.etcd.io/registry/act
create
/discoverygc/2021-11-20T09:09:48Z/5c689d7b17d78fd0bd5de1791fa58734

We can see that the Discovery Server has correctly written the discovery token and related data into ETCD.

I noticed an issue here. The official ETCD documentation states:

By convention the etcd discovery protocol uses the key prefix _etcd/registry. If http://example.com hosts an etcd cluster for discovery service, a full URL to discovery keyspace will be http://example.com/v2/keys/_etcd/registry. We will use this as the URL prefix in the example.

But looking at what’s stored in ETCD, there’s a slight discrepancy in the keys. I currently suspect this might be due to version iterations, and the documentation hasn’t been updated. If anyone knows better, please correct me.

That’s all for the Discovery Server content. In the next issue, I’ll continue to update on the pitfalls encountered when deploying an ETCD cluster.

I hope this is helpful, Happy hacking…