hashicorp / hashicorp/terraform
proposal: Ability to identify and action unmanaged resources
- Dominant language
- Go
- Stars
- 49.7k
- Forks
- 10.6k
- Avg merge
- 21h 30m
- Merged PRs (30d)
- 100
Description
### Current Terraform Version
```
0.14.x (master @ 5677978eb08adddeb414b1a8e36d821b89f8ad42)
```
### Use-cases
Within most Infrastructure as Code (IAC) repositories, the goal is to treat it as the source of truth and manage resources in a consistent way. Unfortunately, this is often an uphill battle as most remote systems have some sort of UI/console where you can add resources outside of the IAC processes. This results in reinventing the wheel over and over again between systems to compare what Terraform manages and what is present in the remote API. The outcome I'm looking for here is a way that providers can opt-in to detect, compare and recify drift in the remote API using Terraform's existing state file.
This would enable operators a way to treat the IAC process as the source of truth and rectify any changes in the same pipeline they may already be using to lint/apply their Terraform changes. This process at a high level also leads to better auditability and known states in the event something goes wrong (intentional or malicious).
### Attempted Solutions
In the past, myself and teams I've worked on have put together CI pipelines that use a combination of `terraform state list` and cURLing the remote APIs to compare identifiers to see what has been created outside of Terraform in order to alert or notify teams. This is pretty tedious as each Terraform resource needs to be checked and depending on the size of the provider, can be quite intensive to complete.
I've also seen a couple of cases where within a provider `Read` method, the full state is checked. This works however it is abuse of the `Read` method and doesn't scale with the number of resources managed.
### Proposal
I'd like to propose a way that providers can opt-in to add a new method to the schema in order to determine differences. Here is a brief example of what I'd think could achieve it:
```go
func resourceExampleResource() *schema.Resource {
return &schema.Resource{
Create: resourceExampleResourceCreate,
Read: resourceExampleResourceRead,
Update: resourceExampleResourceUpdate,
Delete: resourceExampleResourceDelete,
Importer: &schema.ResourceImporter{
State: resourceExampleResourceImport,
},
ResourceCompare: resourceExampleResourceCompare,
Schema: map[string]*schema.Schema{
// .. snip
},
}
}
func resourceExampleResourceCompare(resources) error {
api := meta.(*remote.API)
// fetch all terraform managed resources from state and build up a list of
// known IDs to compare.
terraformResources := d.ReachIntoStateAndReturnAllResources()
managedIDs := []string{}
for _, tfResource := range terraformResources {
managedIDs := append(managedIDs, tfResource.ID)
}
// get all resources in the remote API
remoteResources := api.GetAllResources()
remoteResourceIDs := []string{}
for _, remoteResource := range remoteResources {
remoteResourceIDs := append(remoteResourceIDs, remoteResource.ID)
}
// compare the two lists picking out any uniques that appear in the remote
diff := diffSlices(managedIDs, remoteResourceIDs)
if (len(diff) != 0) {
return errors.New("terraform state is out of sync with remote API: %v", diff)
}
return nil
}
```
The idea of allowing providers to configure how they do the comparison is that they can ecapsulate any logic they need to do the comparison without applying any hard rules from upstream Terraform.
### References
#4728, #21947 (if you squint)
Contributor guide
Research direction
Start by reviewing the proposed provider schema and resource lifecycle in the issue, then read references #4728 and #21947 for prior context. Since no implementation files or tests are named, first determine the framework scope and an actionable design; done would require an agreed provider-facing approach with defined behavior and validation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- devops, infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100