etcd-io / etcd-io/etcd

`etcdutl init`: initialize a member's data directory offline, without a snapshot file or a running server

Open
#22,090 4 comments 1 reaction 0 assignees View on GitHub
Dominant language
Go
Stars
52.3k
Forks
10.5k
Avg merge
2d 21h
Merged PRs (30d)
43

Description

### What would you like to be added?

A new `etcdutl init` command that creates a fully formed etcd data directory (backend db, WAL, membership snapshot) for a member of a new cluster, entirely offline. With no flags it initializes a single-member data directory using the same defaults as the server (`default.etcd`, `http://localhost:2380`):

```console
$ etcdutl init
$ find default.etcd
default.etcd/member/snap/0000000000000001-0000000000000001.snap
default.etcd/member/snap/db
default.etcd/member/wal/0000000000000000-0000000000000000.wal
$ etcd --data-dir default.etcd # starts, no bootstrap flags needed
```

For a multi-member cluster, run it once per member with that member's identity and the shared topology:

```bash
etcdutl init \
--name etcd-0 \
--data-dir /var/lib/etcd \
--initial-cluster etcd-0=http://etcd-0:2380,etcd-1=http://etcd-1:2380,etcd-2=http://etcd-2:2380 \
--initial-cluster-token my-cluster \
--initial-advertise-peer-urls http://etcd-0:2380
```

The produced data directories contain the full cluster membership, so each member afterwards starts with just `--name` and `--data-dir` — no `--initial-*` bootstrap flags on the server invocation.

The command should be **idempotent**: if the data directory already exists, `init` validates that it belongs to a member with the same configuration (member ID derived from the peer URLs and cluster token, member name) and that it passes offline consistency verification, then exits 0 instead of failing or overwriting. A `--no-verify` flag skips the WAL consistency scan for large data directories; identity checks always run.

This is close to what `etcdutl snapshot restore` already does, minus the requirement of having a snapshot file: init synthesizes an empty backend database (standard buckets, current storage version) and reuses the restore machinery to write the WAL and membership.

**Out of scope:** joining members to an *existing* cluster (`member add` / StatefulSet scale-up) — membership changes must go through raft conf changes on the live quorum. `init` covers initial bootstrap only; an unknown member name against an existing data directory fails loudly rather than mutating membership.

### Why is this needed?

Bootstrapping etcd on Kubernetes (and similar orchestrators) currently requires either runtime discovery, an operator, or passing identical `--initial-*` flags to every member and relying on first-start-only semantics — flags that silently become no-ops after the first boot and confuse configuration drift detection.

With an idempotent `etcdutl init`, a StatefulSet expresses bootstrap declaratively — an init container provisions or validates the data directory on every pod start, and the runtime container knows nothing about bootstrap:

```yaml
kind: StatefulSet
spec:
replicas: 3
template:
spec:
initContainers:
- name: init
image: gcr.io/etcd-development/etcd:v3.8.0
command:
- etcdutl
- init
- --name=$(POD_NAME)
- --data-dir=/var/lib/etcd
- --initial-cluster=etcd-0=http://etcd-0.etcd:2380,etcd-1=http://etcd-1.etcd:2380,etcd-2=http://etcd-2.etcd:2380
- --initial-cluster-token=etcd-cluster
- --initial-advertise-peer-urls=http://$(POD_NAME).etcd:2380
volumeMounts:
- name: data
mountPath: /var/lib/etcd
containers:
- name: etcd
image: gcr.io/etcd-development/etcd:v3.8.0
command:
- etcd
- --name=$(POD_NAME)
- --data-dir=/var/lib/etcd
- --listen-peer-urls=http://0.0.0.0:2380
- --listen-client-urls=http://0.0.0.0:2379
- --advertise-client-urls=http://$(POD_NAME).etcd:2379
volumeMounts:
- name: data
mountPath: /var/lib/etcd
```

(`POD_NAME` downward-API env, headless Service, and `volumeClaimTemplates` omitted for brevity.)

Properties this gives operators:

- **Every pod start is the same code path.** First boot initializes; every subsequent boot validates. A wrong volume mount, a changed cluster token, or a renamed member fails loudly in the init container instead of producing a member with a different identity.
- **No bootstrap/runtime flag split.** The server container never carries `--initial-*` flags that are only meaningful on first start.
- **No discovery service, no operator required** for the common static-topology case.

This could also serve as a building block for the official [etcd-operator](https://github.com/etcd-io/etcd-operator) (which currently passes `ETCD_INITIAL_CLUSTER` and `--initial-advertise-peer-urls` to the runtime container) and for Helm charts, which cannot rely on an operator and today resort to entrypoint shell scripts to guess bootstrap state.

The same workflow applies outside Kubernetes: bake data directories into machine images, provision them with configuration management, etc.

Contributor guide

Open the contributing guide

Research direction

Start with the existing `etcdutl snapshot restore` command and its restore machinery, since the issue identifies that as the implementation basis. Trace how it creates the backend, WAL, and membership snapshot, then determine how `etcdutl init` should add empty-database creation, identity validation, idempotency, and optional WAL verification. Done means offline single- and multi-member initialization produces directories that start without bootstrap flags and safely validates repeated runs.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
cli, databases, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.