By invalidating all old etcd discovery tokens you have broken my workflow.

[SleepyBrett]

etcd 3.3.9

In response to coreos/discovery.etcd.io#64 (comment)

Previous workflow.

Create discovery url, create 5 node etcd cluster in asg.

When I want to roll over one of those nodes I do the following:

1. Delete old node (shut down etcd first)

2. on current cluster member remove old member

3. one current cluster member add new member

4. on replacment node create a dropin systemd unit without discovery and with current members and self, daemon-reload, restart etcd

5. confirm success, remove restore droping, daemon-reload

That doesn't work anymore on clusters with 'old' discovery tokens.

instead the new node, with the old discovery token starts up and then crashes:

May 16 16:08:20 ip-172-27-187-218 etcd-wrapper[1170]: 2019-05-16 16:08:20.689784 E | etcdmain: failed to join discovery cluster (discovery: bad discovery endpoint)
May 16 16:08:20 ip-172-27-187-218 etcd-wrapper[1170]: 2019-05-16 16:08:20.689816 I | etcdmain: discovery token https://discovery.etcd.io/9a63c50f66e2803fb5ad005643cb7e60 was used, but failed to bootstrap the cluster.
May 16 16:08:20 ip-172-27-187-218 etcd-wrapper[1170]: 2019-05-16 16:08:20.689822 I | etcdmain: please generate a new discovery token and try to bootstrap again.

[SleepyBrett]

my rollover script was tripping up because the invalid token causes etcd to go into crashloop instead of going into proxy mode. We had scripts checking for proxy mode and just removed them. Works fine now. Still not a great way to transition a service.

journalctl -u etcd | grep "proxy: listening for client requests on https://0.0.0.0:2379" now fails.

[philips]

I apologize that this caused you issues. Unfortunately the etcd community nor the CNCF had the resources to do a more sophisticated migration.