Debugging EBS 12.2 SSL Setup

 Hi, 


Here is article related the setup of TLS on Our EBS 12.2. We have F5 load balancer, it terminates SSL. ie. User to F5 encrypted with TLS and F5 to EBS communication is non-TLS. After following the setup of context variables, all looked good except, forms login providing popups. 

"The Certificate is not valid and cannot be used to verify the identity of this website". We know that we have created new cert and associated with Load Balancer. No other clue on logs of EBS. 



We enabled tracing by setting runtime parameters.

-Djava.security.debug=all -Djavax.net.debug=all




Java Console errors.

Caused by: java.net.ConnectException: Connection refused: connect

        at java.net.DualStackPlainSocketImpl.connect0(Native Method)

        at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source)

        at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)

        at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)

        at java.net.AbstractPlainSocketImpl.connect(Unknown Source)

        at java.net.PlainSocketImpl.connect(Unknown Source)

        at java.net.SocksSocketImpl.connect(Unknown Source)

        at java.net.Socket.connect(Unknown Source)

        at java.net.Socket.connect(Unknown Source)

        at java.net.Socket.<init>(Unknown Source)

        at java.net.Socket.<init>(Unknown Source)

        at com.sun.jndi.ldap.Connection.createSocket(Unknown Source)

        ... 74 more

certpath: Can't create URICertStore: unable to create InitialDirContext using supplied parameters

certpath: Trying to fetch CRL from DP http://pki.example.com/pki/DEV%20Staff%20Issuing%20CA.crl

certpath: CertStore URI:http://pki.example.com/pki/DEV%20Staff%20Issuing%20CA.crl

certpath: Downloading new CRL...

certpath: X509CRLSelector.match: update out-of-range

certpath: Returning 0 CRLs

Allow unsafe renegotiation: false

Allow legacy hello messages: true



On the Java Console, I have collected the logs and reviewed. it is definitely to do something with CRLs. I have contacted Security team. It took a while to figure out the cause. Finally it was expired CRL (Certificate Revocation List) file for Dev Staff Issuing CA causing the problem. The updated CRL has been published to the pki.example.com web server and CRL validations with the full chain are working now.


Hooray!!! Happy Debugging!!

Terraform - RDS Tainted

Hi Everyone


While working on one of the DB build, encountered this tained issue with IaaC terraforming. 

It was mysql community engines. terraform had both 8.0 version and 5.7 version of mysql. 

terraform plan was coming up with the output that it has be destroyed and rebuilt. Ah... it is running 

prod database, we do NOT want it to perform that redeploy operation. Error messages are cryptic. 


aws_route53_record.dnsrecord1: Refreshing state... [id=Z08109741XXXXA3L0CSZJ_wordpress.db.example.com_CNAME]


Terraform used the selected providers to generate the following execution

plan. Resource actions are indicated with the following symbols:

  ~ update in-place

-/+ destroy and then create replacement


Terraform will perform the following actions:


  # aws_db_instance.materialsdbprd-instance is tainted, so must be replaced

-/+ resource "aws_db_instance" "materialsdbprd-instance" {

      ~ address                               = "tech-materialsprd.xxxxx.us-east-1.rds.amazonaws.com" -> (known after apply)

      ~ arn                                   = "arn:aws:rds:us-east-1:9999999999:db:edtech-materialsprd" -> (known after apply)

      ~ backup_retention_period               = 14 -> (known after apply)

      ~ backup_target                         = "region" -> (known after apply)

      ~ backup_window                         = "08:00-08:30" -> (known after apply)

      ~ ca_cert_identifier                    = "rds-ca-2019" -> "rds-ca-rsa2048-g1"

      + character_set_name                    = (known after apply)

      - customer_owned_ip_enabled             = false -> null

      ~ db_name                               = "materialsdb" -> (known after apply)

      ~ endpoint                              = "tech-materialsprd.xxxxxx.us-east-1.rds.amazonaws.com:3306" -> (known after apply)

      ~ engine_version_actual                 = "5.7.42" -> (known after apply)

      + final_snapshot_identifier             = "materialsdbprd-instance-b4drop-bkp"

      ~ hosted_zone_id                        = "Z2RXXXXXXX61AM" -> (known after apply)

      - iam_database_authentication_enabled   = false -> null

      ~ id                                    = "db-XXXXXXXDXZDY" -> (known after apply)

      + identifier_prefix                     = (known after apply)

      ~ iops                                  = 0 -> (known after apply)

      + kms_key_id                            = (known after apply)

      ~ latest_restorable_time                = "2024-01-03T20:00:00Z" -> (known after apply)

      ~ license_model                         = "general-public-license" -> (known after apply)

      ~ listener_endpoint                     = [] -> (known after apply)

      ~ maintenance_window                    = "mon:09:00-mon:09:30" -> (known after apply)

      ~ master_user_secret                    = [] -> (known after apply)

      + master_user_secret_kms_key_id         = (known after apply)

      ~ max_allocated_storage                 = 0 -> 1000

      + monitoring_role_arn                   = (known after apply)

      + nchar_character_set_name              = (known after apply)

      ~ network_type                          = "IPV4" -> (known after apply)

      ~ performance_insights_kms_key_id       = "arn:aws:kms:us-east-1:99999999:key/e6dd4b7c-3359-404xxxxxxxxcee697f" -> (known after apply)

      + replica_mode                          = (known after apply)

      ~ replicas                              = [] -> (known after apply)

      ~ resource_id                           = "db-WQJAZ4XXXXXXXZDY" -> (known after apply)

      ~ status                                = "available" -> (known after apply)

      - storage_encrypted                     = false -> null

      ~ storage_throughput                    = 0 -> (known after apply)

      ~ storage_type                          = "standard" -> "gp3"

        tags                                  = {

            "ApplicationKey" = "multimedia"

            "Automation"     = "terraform"

            "CreatedBy"      = "admin@example.com"

            "ModuleKey"      = "tech"

        }

      + timezone                              = (known after apply)

        # (27 unchanged attributes hidden)


        # (1 unchanged block hidden)

    }


  # aws_route53_record.dnsrecord2 will be updated in-place

  ~ resource "aws_route53_record" "dnsrecord2" {

        id                               = "ZXXXXXXXX_materials.db.example.com_CNAME"

        name                             = "material.db.example.com"

      ~ records                          = [

          - "tech-materialsprd.xxxxxx.us-east-1.rds.amazonaws.com",

        ] -> (known after apply)

        # (5 unchanged attributes hidden)

    }

Plan: 1 to add, 1 to change, 1 to destroy.

------------------------------------------------------------------------

Cost estimation:

Resources: 3 of 7 estimated

           $243.61600000000001976/mo +$0.30000000000000528

------------------------------------------------------------------------

Organization policy check:


$BitBucketRepos\multimedia-edtech-dbprod-terraform> terraform  untaint    aws_db_instance.materialsdbprd-instance


│ Error: Error acquiring the state lock

│ Error message: resource not found

│ Terraform acquires a state lock to protect the state from being written
│ by multiple users at the same time. Please resolve the issue above and try
│ again. For most commands, you can disable locking with the "-lock=false"
│ flag, but this is not recommended.


$terraform untaint  aws_db_instance.materialsdbprd-instance


$BitBucketRepos\multimedia-edtech-dbprod-terraform> terraform plan

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration

and found no differences, so no changes are needed.


After untainting, code is in clean slate. good to go and perform pull request to merge with master branch. Had KT session and handed over the code to product team manage.