Terraform - RDS Tainted

Hi Everyone


While working on one of the DB build, encountered this tained issue with IaaC terraforming. 

It was mysql community engines. terraform had both 8.0 version and 5.7 version of mysql. 

terraform plan was coming up with the output that it has be destroyed and rebuilt. Ah... it is running 

prod database, we do NOT want it to perform that redeploy operation. Error messages are cryptic. 


aws_route53_record.dnsrecord1: Refreshing state... [id=Z08109741XXXXA3L0CSZJ_wordpress.db.example.com_CNAME]


Terraform used the selected providers to generate the following execution

plan. Resource actions are indicated with the following symbols:

  ~ update in-place

-/+ destroy and then create replacement


Terraform will perform the following actions:


  # aws_db_instance.materialsdbprd-instance is tainted, so must be replaced

-/+ resource "aws_db_instance" "materialsdbprd-instance" {

      ~ address                               = "tech-materialsprd.xxxxx.us-east-1.rds.amazonaws.com" -> (known after apply)

      ~ arn                                   = "arn:aws:rds:us-east-1:9999999999:db:edtech-materialsprd" -> (known after apply)

      ~ backup_retention_period               = 14 -> (known after apply)

      ~ backup_target                         = "region" -> (known after apply)

      ~ backup_window                         = "08:00-08:30" -> (known after apply)

      ~ ca_cert_identifier                    = "rds-ca-2019" -> "rds-ca-rsa2048-g1"

      + character_set_name                    = (known after apply)

      - customer_owned_ip_enabled             = false -> null

      ~ db_name                               = "materialsdb" -> (known after apply)

      ~ endpoint                              = "tech-materialsprd.xxxxxx.us-east-1.rds.amazonaws.com:3306" -> (known after apply)

      ~ engine_version_actual                 = "5.7.42" -> (known after apply)

      + final_snapshot_identifier             = "materialsdbprd-instance-b4drop-bkp"

      ~ hosted_zone_id                        = "Z2RXXXXXXX61AM" -> (known after apply)

      - iam_database_authentication_enabled   = false -> null

      ~ id                                    = "db-XXXXXXXDXZDY" -> (known after apply)

      + identifier_prefix                     = (known after apply)

      ~ iops                                  = 0 -> (known after apply)

      + kms_key_id                            = (known after apply)

      ~ latest_restorable_time                = "2024-01-03T20:00:00Z" -> (known after apply)

      ~ license_model                         = "general-public-license" -> (known after apply)

      ~ listener_endpoint                     = [] -> (known after apply)

      ~ maintenance_window                    = "mon:09:00-mon:09:30" -> (known after apply)

      ~ master_user_secret                    = [] -> (known after apply)

      + master_user_secret_kms_key_id         = (known after apply)

      ~ max_allocated_storage                 = 0 -> 1000

      + monitoring_role_arn                   = (known after apply)

      + nchar_character_set_name              = (known after apply)

      ~ network_type                          = "IPV4" -> (known after apply)

      ~ performance_insights_kms_key_id       = "arn:aws:kms:us-east-1:99999999:key/e6dd4b7c-3359-404xxxxxxxxcee697f" -> (known after apply)

      + replica_mode                          = (known after apply)

      ~ replicas                              = [] -> (known after apply)

      ~ resource_id                           = "db-WQJAZ4XXXXXXXZDY" -> (known after apply)

      ~ status                                = "available" -> (known after apply)

      - storage_encrypted                     = false -> null

      ~ storage_throughput                    = 0 -> (known after apply)

      ~ storage_type                          = "standard" -> "gp3"

        tags                                  = {

            "ApplicationKey" = "multimedia"

            "Automation"     = "terraform"

            "CreatedBy"      = "admin@example.com"

            "ModuleKey"      = "tech"

        }

      + timezone                              = (known after apply)

        # (27 unchanged attributes hidden)


        # (1 unchanged block hidden)

    }


  # aws_route53_record.dnsrecord2 will be updated in-place

  ~ resource "aws_route53_record" "dnsrecord2" {

        id                               = "ZXXXXXXXX_materials.db.example.com_CNAME"

        name                             = "material.db.example.com"

      ~ records                          = [

          - "tech-materialsprd.xxxxxx.us-east-1.rds.amazonaws.com",

        ] -> (known after apply)

        # (5 unchanged attributes hidden)

    }

Plan: 1 to add, 1 to change, 1 to destroy.

------------------------------------------------------------------------

Cost estimation:

Resources: 3 of 7 estimated

           $243.61600000000001976/mo +$0.30000000000000528

------------------------------------------------------------------------

Organization policy check:


$BitBucketRepos\multimedia-edtech-dbprod-terraform> terraform  untaint    aws_db_instance.materialsdbprd-instance


│ Error: Error acquiring the state lock

│ Error message: resource not found

│ Terraform acquires a state lock to protect the state from being written
│ by multiple users at the same time. Please resolve the issue above and try
│ again. For most commands, you can disable locking with the "-lock=false"
│ flag, but this is not recommended.


$terraform untaint  aws_db_instance.materialsdbprd-instance


$BitBucketRepos\multimedia-edtech-dbprod-terraform> terraform plan

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration

and found no differences, so no changes are needed.


After untainting, code is in clean slate. good to go and perform pull request to merge with master branch. Had KT session and handed over the code to product team manage. 

No comments: