Introduction⌗

Our business often needs to generate URLs for resources across multiple projects, and due to environment consistency issues, we frequently encounter situations where URLs generated in the local development environment use the HTTPS protocol.

In the production environment, we use annually purchased certificates deployed on cloud SLBs. In the test environment, we originally used HTTP protocol uniformly, which is why many issues couldn’t be detected in the test environment but were exposed after deployment to production. Later, we switched to using free certificates issued by Let’s Encrypt!

There’s still a problem where team members use different domains and protocols in their local development environments, which increases communication costs.

Now we want to use a Docker environment to simulate the production environment as much as possible!

Solution Research⌗

After research, we found the following open-source CA projects:

After initial understanding and hands-on experience, we found that Step Certificates has the most comprehensive documentation and relatively good user experience, although it has a certain learning curve and requires some understanding of X.509.

Installation⌗

It’s recommended to first read through the official documentation to form a knowledge framework. Then combine it with the official blog: Run your own private CA & ACME server using step-ca for implementation!

Docker⌗

If we want to use it for team internal basic development environment setup, we must use it in containers:

services:
  step-ca:
    image: smallstep/step-ca:latest
    labels:
      - traefik.enable=false
    restart: always
    volumes:
      - step-ca:/home/step
    hostname: step-ca
    networks:
      traefik:
        ipv4_address: 10.8.10.254
    extra_hosts:
      - ca.svc.dev:127.0.0.1
    environment:
      - TZ=Asia/Shanghai
      - DOCKER_STEPCA_INIT_NAME=Smallstep
      - DOCKER_STEPCA_INIT_ACME=true
      - DOCKER_STEPCA_INIT_ADDRESS=0.0.0.0:443
      - DOCKER_STEPCA_INIT_PASSWORD=ECkFB7TwcHVPuLhOtKwCIP3J3pNGHarF
      - DOCKER_STEPCA_INIT_DNS_NAMES=ca.svc.dev,acme-v02.api.letsencrypt.org,step-ca
      - [email protected]
      - DOCKER_STEPCA_INIT_REMOTE_MANAGEMENT=true
    container_name: step-ca

volumes:
  step-ca:
    name: step-ca

networks:
  traefik:
    external: true

Configuration notes:

Line 12: Set a fixed IP for the container to facilitate requests from other services
Line 14: Create local mapping for HTTPS domain to avoid container self-check failure

Currently, we’ve found that even though DOCKER_STEPCA_INIT_ACME is set to true in the environment variables, it still cannot automatically register the ACME Provider. After the service is up, we need to modify the configuration file in the container’s mounted volume!

docker compose up -d
docker compose logs
step-ca  | Generating root certificate... done!
step-ca  | Generating intermediate certificate... done!
step-ca  | 
step-ca  | ✔ Root certificate: /home/step/certs/root_ca.crt
step-ca  | ✔ Root private key: /home/step/secrets/root_ca_key
step-ca  | ✔ Root fingerprint: ab8462fc8e67c581626e829a18f5a801b5eea15baf07ee943a6f2bfa24ee8e40
step-ca  | ✔ Intermediate certificate: /home/step/certs/intermediate_ca.crt
step-ca  | ✔ Intermediate private key: /home/step/secrets/intermediate_ca_key
step-ca  | badger 2024/07/16 06:37:16 INFO: All 0 tables opened in 0s
step-ca  | badger 2024/07/16 06:37:16 INFO: Storing value log head: {Fid:0 Len:30 Offset:3321}
step-ca  | badger 2024/07/16 06:37:16 INFO: [Compactor: 173] Running compaction: {level:0 score:1.73 dropPrefixes:[]} for level: 0
step-ca  | badger 2024/07/16 06:37:16 INFO: LOG Compact 0->1, del 1 tables, add 1 tables, took 5.842993ms
step-ca  | badger 2024/07/16 06:37:16 INFO: [Compactor: 173] Compaction for level: 0 DONE
step-ca  | badger 2024/07/16 06:37:16 INFO: Force compaction on level 0 done
step-ca  | ✔ Database folder: /home/step/db
step-ca  | ✔ Default configuration: /home/step/config/defaults.json
step-ca  | ✔ Certificate Authority configuration: /home/step/config/ca.json
step-ca  | ✔ Admin provisioner: admin (JWK)
step-ca  | ✔ Super admin subject: [email protected]
step-ca  | 
step-ca  | Your PKI is ready to go. To generate certificates for individual services see 'step help ca'.
step-ca  | 
step-ca  | FEEDBACK 😍 🍻
step-ca  |   The step utility is not instrumented for usage statistics. It does not phone
step-ca  |   home. But your feedback is extremely valuable. Any information you can provide
step-ca  |   regarding how you're using `step` helps. Please send us a sentence or two,
step-ca  |   good or bad at [email protected] or join GitHub Discussions
step-ca  |   https://github.com/smallstep/certificates/discussions and our Discord
step-ca  |   https://u.step.sm/discord.
step-ca  | 
step-ca  | 👉 Your CA administrative username is: [email protected]
step-ca  | 👉 Your CA administrative password is: ECkFB7TwcHVPuLhOtKwCIP3J3pNGHarF
step-ca  | 🤫 This will only be displayed once.
step-ca  | badger 2024/07/16 06:37:16 INFO: All 1 tables opened in 1ms
step-ca  | badger 2024/07/16 06:37:16 INFO: Replaying file id: 0 at offset: 3351
step-ca  | badger 2024/07/16 06:37:16 INFO: Replay took: 1.708µs
step-ca  | 2024/07/16 06:37:16 Building new tls configuration using step-ca x509 Signer Interface
step-ca  | 2024/07/16 06:37:16 Starting Smallstep CA/0.27.1 (linux/arm64)
step-ca  | 2024/07/16 06:37:16 Documentation: https://u.step.sm/docs/ca
step-ca  | 2024/07/16 06:37:16 Community Discord: https://u.step.sm/discord
step-ca  | 2024/07/16 06:37:16 Config file: /home/step/config/ca.json
step-ca  | 2024/07/16 06:37:16 The primary server URL is https://ca.svc.dev:443
step-ca  | 2024/07/16 06:37:16 Root certificates are available at https://ca.svc.dev:443/roots.pem
step-ca  | 2024/07/16 06:37:16 Additional configured hostnames: acme-v02.api.letsencrypt.org, step-ca
step-ca  | 2024/07/16 06:37:16 X.509 Root Fingerprint: ab8462fc8e67c581626e829a18f5a801b5eea15baf07ee943a6f2bfa24ee8e40
step-ca  | 2024/07/16 06:37:16 Serving HTTPS on 0.0.0.0:443 ...

Configuration⌗

After the container starts, Smallstep will generate the corresponding configuration:

tree
.
├── certs
│   ├── intermediate_ca.crt
│   └── root_ca.crt
├── config
│   ├── ca.json
│   └── defaults.json
├── db
│   ├── 000000.vlog
│   ├── 000030.sst
│   ├── KEYREGISTRY
│   └── MANIFEST
├── secrets
│   ├── intermediate_ca_key
│   ├── password
│   └── root_ca_key
└── templates

6 directories, 11 files

Among these, the most commonly used are certs/root_ca.crt and config/ca.json. Below is the configuration of config/ca.json:

{
  "root": "/home/step/certs/root_ca.crt",
  "federatedRoots": null,
  "crt": "/home/step/certs/intermediate_ca.crt",
  "key": "/home/step/secrets/intermediate_ca_key",
  "address": "0.0.0.0:443",
  "insecureAddress": "",
  "dnsNames": [
    "ca.svc.dev",
    "acme-v02.api.letsencrypt.org",
    "step-ca"
  ],
  "logger": {
    "format": "text"
  },
  "db": {
    "type": "badgerv2",
    "dataSource": "/home/step/db",
    "badgerFileLoadingMode": ""
  },
  "authority": {
    "claims": {
      "disableRenewal": false,
      "minTLSCertDuration": "2160h",
      "maxTLSCertDuration": "2160h",
      "defaultTLSCertDuration": "2160h",
      "allowRenewalAfterExpiry": false,
      "minHostSSHCertDuration": "5m",
      "maxHostSSHCertDuration": "1680h",
      "minUserSSHCertDuration": "5m",
      "maxUserSSHCertDuration": "24h",
      "defaultUserSSHCertDuration": "16h",
      "defaultHostSSHCertDuration": "720h"
    },
    "enableAdmin": true,
    "provisioners": [
      {
        "type": "ACME",
        "name": "acme"
      }
    ]
  },
  "tls": {
    "cipherSuites": [
      "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256",
      "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256"
    ],
    "minVersion": 1.2,
    "maxVersion": 1.3,
    "renegotiation": false
  }
}

Configuration notes:

Lines 22-34 in the claims section are manually added configurations, mainly for setting certificate validity duration, etc.
Lines 36-41 in the provisioners section are used to enable ACME-related configurations

One thing to note here is that the official blog mentions that minTLSCertDuration and maxTLSCertDuration should be configured within the ACME Provider, but in actual testing, this doesn’t work!

After modifying the configuration, restart the container!

Host Machine⌗

To use the CA on the host machine, you need to add the CA’s root certificate to the system’s trusted certificate list. On macOS, you can use the following command to install Smallstep:

brew install step
step certificate install <ROOT_CA_PATH>

Replace the <ROOT_CA_PATH> part with the file path of the root_ca.crt certificate in your container volume, then execute the installation!

After successful installation, the system will be able to trust all certificates issued by Smallstep!

Manually Generate Certificates⌗

docker compose exec step-ca step ca certificate localhost certs/localhost.crt certs/localhost.key
✔ Provisioner: acme (ACME)
Using Standalone Mode HTTP challenge to validate localhost . done!
Waiting for Order to be 'ready' for finalization .. done!
Finalizing Order .. done!
✔ Certificate: certs/localhost.crt
✔ Private Key: certs/localhost.key

After executing the command, you will be prompted to select a Provider; choose ACME here.

The generated certificate files are in the certs directory of the step-ca volume. You can copy the certificates to the host machine and verify them with the step CLI:

step certificate inspect localhost.crt
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 208228708681982736082256753293896422484 (0x9ca763f9fb606e7dfb51d17c15a39454)
    Signature Algorithm: ECDSA-SHA256
        Issuer: O=Smallstep,CN=Smallstep Intermediate CA
        Validity
            Not Before: Jul 18 01:54:16 2024 UTC
            Not After : Oct 16 01:55:16 2024 UTC
        Subject: CN=localhost
        Subject Public Key Info:
            Public Key Algorithm: ECDSA
                Public-Key: (256 bit)
                X:
                    bf:93:74:5a:7d:75:a1:af:68:3d:52:ec:0b:8f:66:
                    09:a7:40:bb:94:93:7c:f8:a0:ad:03:be:e2:ec:51:
                    11:7c
                Y:
                    52:29:b1:c5:a1:c2:0b:c8:97:c3:da:65:1b:36:2a:
                    d9:4f:2d:2f:48:f9:b3:c5:8e:f5:4b:9d:40:35:7e:
                    a4:5e
                Curve: P-256
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature
            X509v3 Extended Key Usage:
                Server Authentication, Client Authentication
            X509v3 Subject Key Identifier:
                3C:E8:3A:68:B4:4A:C2:63:E9:7E:6A:0C:8A:29:F7:2A:CF:38:94:D2
            X509v3 Authority Key Identifier:
                keyid:83:CC:31:DD:68:E3:02:76:25:E1:50:B9:FD:15:59:81:29:6C:EB:96
            X509v3 Subject Alternative Name:
                DNS:localhost
            X509v3 Step Provisioner:
                Type: ACME
                Name: acme
    Signature Algorithm: ECDSA-SHA256
         30:44:02:20:20:95:51:3c:14:72:1e:9c:14:7f:3b:8a:4f:5b:
         e9:4a:24:a1:a4:b1:74:87:dc:20:81:2c:68:b6:d2:5b:d2:b3:
         02:20:7c:19:f0:df:8f:c0:ef:f6:d0:9c:01:69:d0:4a:ca:e1:
         39:d7:67:e7:71:4f:9a:2b:25:2a:79:a6:87:e6:f9:ec

Conclusion⌗

At this point, the PKI based on Smallstep has been set up. In this chapter, we only explained how to manually generate certificates. If we only need to manually generate certificates, then Smallstep is much more complex compared to the CA CLI tools mentioned at the beginning of the article!

But the core of Smallstep is its built-in CA service and support for the ACME protocol!

There are still many areas that need to be integrated with existing tools, such as how Traefik can use Smallstep to automatically apply for HTTPS certificates…

I hope this is helpful, Happy hacking…

Setting Up a Private ACME Server with Step-CA