Protection and tools for managing stateful resources and avoiding deletion

RFC (Rev 1.)

Here's a proposal for a simple mechanism to prevent damage caused by logical name mapping changes.

If a resource is marked as stateful, it's Terraform resource name will be determined by looking up its path in a map stored in a <entrypoint>.w.lock file, next to the entrypoint file, which will be committed to the repository.

If an entry for this resource cannot be found in the lockfile, the compilation will fail and instruct the user to explicitly add an entry and provide a resource name.

Additionally, compilation will also fail if the file includes an entry that doesn't map to a stateful resource in the app.

This will ensure that:

1. Stateful resources are explicitly named.
2. If a stateful resource is moved within the app (e.g. it's path is changed), then users will easily be able to map the new path to the same stateful resource.
3. If a stateful resource is accidentally removed, users will have to acknowledge this by explicitly removing the resource entry from the lockfile.

We can offer some CLI commands to edit the lockfile, but that's not P1.

Here's an example.

Say I write this Wing called hello.main.w:

bring cloud;

new cloud.Bucket();

Now, I compile:

$ wing compile -t tf-aws hello.main.w
ERROR: stateful resource "/root/Bucket" doesn't have a name in the lockfile.
Please edit `hello.main.w.lock`.
$ cat hello.main.w.lock
/root/Bucket: <NEW>
$ edit hello.main.w.lock
/root/Bucket: my_s3_bucket
$ wing compile -t tf-aws hello.main.w
$ # cool!

Now, let's say we refactored the app and we will move the same bucket into a class:

bring cloud;

class MyStore {
  new() {
    new cloud.Bucket();
  }
}

new MyStore();

$ wing compile -t tf-aws hello.main.w
ERROR: stateful resource "/root/MyStore/Bucket" doesn't have a name in the lockfile.
ERROR: stateful resource "/root/Bucket" with the name "my_s3_bucket" was deleted.
Please edit `hello.main.w.lock`.
$ cat hello.main.w.lock
/root/Bucket: my_s3_bucket <DELETED>
/root/MyStore/Bucket: <NEW>
$ edit hello.main.w.lock
/root/MyStore/Bucket: my_s3_bucket
$ wing compile -t tf-aws hello.main.w

At some point we can be smarter and offer a nice DX:

It seems like the stateful resource "my_s3_bucket" was moved
from "/root/Bucket" to "/root/MyStore/Bucket".
Type YES to accept: YES_

We can also offer some CLI commands to update:

$ wing state new /root/Bucket my_s3_bucket
$ wing state mv /root/Bucket /root/MyStore/Bucket
$ wing state rm /root/Bucket
$ # etc

---

Original Feature Spec

Each Wing resource has a path that represents its unique address within the resource tree. This path is used to produce a deterministic Terraform identifier for each resource (which is what Terraform uses in its state file to map to the physical resource).

When refactoring code and resources are moved around, their Terraform identifier could change. In certain cases, especially for stateful resources with important data, this could be hazardous.

Wing resources can be explicitly marked as "stateful" or "stateless". If a resource is marked as "stateful", Wing will protect it from being accidentally deleted during deployment, and will offer a way to associated the

We provide some mechanism for you to relocate the terraform identifier of a resource in order to link the old identifier to the newly placed resource.

Use Cases

In some cases, users may want their Buckets to be emptied when their app is destroyed, while in other cases users may want their Buckets to be retained if their app is destroyed. (And in further cases, users may want to configure this on a per-Bucket basis).

The same general pattern applies to any stateful resource, including Counter etc.

Implementation Notes

Sadly, and surprisingly, Terraform's prevent_destory attribute cannot be used to implement this feature:

Since this argument must be present in configuration for the protection to apply, note that this setting does not prevent the remote object from being destroyed if the resource block were removed from configuration entirely: in that case, the prevent_destroy setting is removed along with it, and so Terraform will allow the destroy operation to succeed.

This means that if the resource is completely removed from the configuration, it will be destroyed in the next apply.

There is a heated conversation in this issue.

Alternatives to consider:

1. Use resource-specific capabilities. It seems like some providers offer explicit deletion protection (e.g sql_database_instance).
2. Add a capability into Wing that will assist with tracking resource identifiers throughout iterations. Something like taking and storing snapshot during compilation that can be used during synthesis to identify/handle these cases.

Intuitively it feels like this is something Wing should handle regardless of the provisioning system, so I believe it's worth trying to find the right mechanism.