DXE-4199 Provider behaviour on terraform refresh when real resources were destroyed

[hightoxicity]

Hi there,

We currently get errors from the provider when true infrastructure has been destroyed into Akamai GTM and out of terraform scope (it means terraform state is no more in sync with reality). it seems like Terraform expects smooth resource reconciliation by the provider when doing refresh (on terraform refresh and included by default into terraform plan)

Terraform and Akamai Terraform Provider Versions

Last release (6.4.0) and all previous one

Affected Resource(s)

All GTM resources, I think the behaviour comes from the provider implementation not main Terraform.

Expected Behavior

https://discuss.hashicorp.com/t/reconcile-destroyed-resources/14207

Refresh behaviour (https://developer.hashicorp.com/terraform/cli/commands/refresh) should lead to the best as possible drift detection and correction, it means we should not face a "Domain Read error", "property Read failed", ... The provider code should track the resource absence and update the state to remove the absent resource. For now, it triggers errors.

Actual Behavior

The provider triggers errors on API read attempts: logger.Errorf("Property Read failed: GetProperty error: %s", err.Error()) into pkg/providers/gtm/resource_akamai_gtm_property.go for example.

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

1. Create an akamai GTM resource (a domain for example) using Terraform (edited on 2024-09-12 at 2:30pm UTC)
2. Destroy the real domain through Akamai web UI
3. Run a terraform refresh or terraform plan then face the issue

References

https://discuss.hashicorp.com/t/reconcile-destroyed-resources/14207

[ckulinsk]

Hello @hightoxicity

Thank you for reporting this issue. I have been able to reproduce it, but I have a couple of follow up questions:

1. Why the resource, that is managed by the terraform, was destroyed using UI?
2. Is using command terraform state rm 'example' not sufficient in this situation?

best regards,
Cyryl

[hightoxicity]

Hi @ckulinsk, we have a lot of resources managed into a single terraform state (for historical reasons), things are updated with terraform into some ADO pipeline, for our production environment it is not rare this pipeline takes many minutes to run (time to trigger, time to terraform plan which may take a couple of minutes mainly due to state refresh, there is a plan computation into one stage then manual approval and apply one to get the planned artifact applied (our way to get something atomical) but because some of our people have based the failover strategy on GTM, this source of truth becomes problematic to solve urgent issues (even more if you cumulate it with DNS TTLs), this is why in urgent cases people are shortcutting the automation and for this reason it leads me to design/implement some pipeline able to reconcile tf state + tf inputs (to make all one happy).

I would like to use the terraform plan output to get what is about to be modified and reconcile only those entities, but I face more difficulties when the plan is failing not been able to refresh state because something was deleted at the end.
I am currently writing something that browse the terraform state list output and map things to full domain spec extraction using API, but since I found a provider should do it natively (remove from state what was deleted at end when refreshing) then I decided to ask you for that to gain also time on reconciliation process.

I hope I provided enough context.
Thanks.

[ckulinsk]

Hello @hightoxicity

thank you for adding more details, we will investigate potential solution and go back to you.

Best regards,
Cyryl