AWS Cloud: Archiving RMAN Backup to S3 Glacier

Hello, 

I have recently got a requirement to decommission the database, however I had to keep the data for ~5 years in archive for legal and compliance purpose. I have decided to keep the backup in S3 glacier deep archive is very reliable and cost effective platform offered by AWS. Here is article to help you move Oracle Backup to AWS S3 Glacier Deep Archive. 

You have option to use Oracle Secure Backup Client modules to move data directly to AWS S3. However it costs $$. Secure Cloud is priced in number of channels that you use for backup/restore. I have explored that option with my Oracle Sales, we didn't pursue that route. Instead I decided to Use NAS to hold RMAN backup sets and copy after that using aws cli commands.

My Environment setup: Linux, Oracle 12c, prod

Tar archive of OH

/oracle_shared/IPDPROD2AWS/

tar -czvpf ipdprod_oh.tar.gz  /u01/app/oracle/product/12.1.0/dbhome_1

rman target=/ |tee -a rman_compressed_bkp.log

RMAN compressed backup execution 

$> cat rman_compressed.rcv

run

{

    allocate channel ch01 device type disk format '/oracle_shared/IPDPROD2AWS/%U' MAXPIECESIZE = 4500m;

    allocate channel ch02 device type disk format '/oracle_shared/IPDPROD2AWS/%U' MAXPIECESIZE = 4500m;

    allocate channel ch03 device type disk format '/oracle_shared/IPDPROD2AWS/%U' MAXPIECESIZE = 4500m;

    allocate channel ch04 device type disk format '/oracle_shared/IPDPROD2AWS/%U' MAXPIECESIZE = 4500m;

    allocate channel ch05 device type disk format '/oracle_shared/IPDPROD2AWS/%U' MAXPIECESIZE = 4500m;

    backup as compressed backupset incremental level 0   database tag 'fullcompressed_ipdprod' INCLUDE CURRENT CONTROLFILE  plus archivelog tag 'arch_compressbkp_IPDPROD';

}

If your backup set has more than 5G, you have deal the upload with multipart upload. Restricted backup set size to 4500MBs to avoid 5GB multipart uploads. 

I have collected the init.ora backup as well. Next I have installed AWS CLI

As a root

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"

cd aws

./install

aws --version

Create S3 Bucket with lifecyle management to move the backup pieces to Glacier deep archive and set expiration to 5 years(1825 days).

I have used terraform commands to achieve that, main.tf contains below.

terraform {

    required_providers {

        aws = {

            source = "hashicorp/aws"

            version = "~> 3.0"

        }

    }

} 

provider "aws" {

    region = "us-east-1"

}

resource "aws_s3_bucket" "S3Bucket" {

    bucket = "oracle-prod-ipdprod-decomfinalbkp"

    acl    = "private"

    tags = {

      Automation = "terraform"

      CreatedBy = "first.last@domain.com"

      Team = "teamnamegoeshere@domain.com"

    }

    versioning {

           enabled    = false

    }

    

    lifecycle_rule {

    id      = "push2glacierdeeparchive"

    enabled = true

    transition {

      days          = 0

      storage_class = "DEEP_ARCHIVE"

    }

    expiration {

      days = 1825

    }

  }

}

 

Generated upload commands

 cd /oracle_shared/IPDPROD2AWS/

ls -1 |awk '{print "nohup aws s3 cp " $1" s3://oracle-financials-ipddb-decomfinalbkp  |tee -a s3_upload.log &;wait"}'>upload.sh

chmod 700 upload.sh

nohup ./upload.sh &

For restore used API calls.

Restore object API call has been made.

aws s3 ls s3://oracle-financials-ipddb-decomfinalbkp | awk '

{print $4}

' | xargs -L 1 aws s3api restore-object --restore-request Days=5 --bucket oracle-financials-ipddb-decomfinalbkp --key

Copied back the RMAN files to NAS location

 cd /oracle_shared/IPDPROD2AWSrestore

nohup  aws s3 sync s3://oracle-financials-ipddb-decomfinalbkp  . --force-glacier-transfer  > restore_11242021.log&

There is a cost involvement in Storing and Retrieving with S3. Be mindful of that if you are playing with test environment. All worked well. Hope this helps someone.