---
title: "Backup & Disaster Recovery"
canonical: "https://kb.uconn.edu/space/IKB/28729212960/Backup%20%26%20Disaster%20Recovery"
format: markdown
---
Accidents happen — pods crash, storage gets corrupted, someone deletes the wrong resource. This guide covers how to back up your OpenShift applications, databases, and configurations so you can recover quickly.

---

## What to Back Up

| Component | Why | How Often |
| --- | --- | --- |
| **Application source code** | Foundation of your app | Every commit (Git handles this) |
| **Container images** | Built artifacts | Stored in registry automatically |
| **ConfigMaps & Secrets** | Environment config, credentials | Before and after changes |
| **Persistent Volume data** | Database files, uploads, user data | Daily or more frequently |
| **Deployment YAML** | Resource definitions | After every change |
| **Routes & Services** | Networking config | After changes |

> **Key Principle:** If it's not in Git and not backed up, assume you'll lose it.

---

## Exporting Resource Definitions

### Export All Resources in a Project

```shell
# Export everything (deployments, services, routes, configmaps, secrets, PVCs)
oc get all,configmap,secret,pvc,route,serviceaccount,rolebinding \
  -o yaml > my-project-backup.yaml

# Export specific resource types
oc get deployment -o yaml > deployments.yaml
oc get configmap -o yaml > configmaps.yaml
oc get secret -o yaml > secrets.yaml
oc get route -o yaml > routes.yaml
oc get pvc -o yaml > pvcs.yaml
```

### Export a Single Resource

```shell
# Export a specific deployment
oc get deployment/my-app -o yaml > my-app-deployment.yaml

# Export a specific secret
oc get secret/my-db-credentials -o yaml > my-db-credentials.yaml
```

### Restore from Exported YAML

```shell
# Apply all resources from a backup file
oc apply -f my-project-backup.yaml

# Apply a specific resource
oc apply -f my-app-deployment.yaml
```

> **Warning:** Exported YAML includes metadata like `resourceVersion` and `uid`. When restoring to a different project, you may need to remove these fields or use `oc create` instead of `oc apply`.

---

## Backing Up Databases

### PostgreSQL Backup

```shell
# Get the pod name
oc get pods -l app=postgresql

# Run pg_dump inside the pod
oc exec postgresql-1-abc123 -- \
  pg_dump -U $POSTGRESQL_USER $POSTGRESQL_DATABASE \
  > backup-$(date +%Y%m%d).sql

# Compressed backup
oc exec postgresql-1-abc123 -- \
  pg_dump -U $POSTGRESQL_USER $POSTGRESQL_DATABASE -Fc \
  > backup-$(date +%Y%m%d).dump
```

### PostgreSQL Restore

```shell
# Restore from SQL dump
oc exec -i postgresql-1-abc123 -- \
  psql -U $POSTGRESQL_USER $POSTGRESQL_DATABASE \
  < backup-20250115.sql

# Restore from compressed dump
oc exec -i postgresql-1-abc123 -- \
  pg_restore -U $POSTGRESQL_USER -d $POSTGRESQL_DATABASE \
  < backup-20250115.dump
```

### MySQL Backup

```shell
# Get the pod name
oc get pods -l app=mysql

# Run mysqldump inside the pod
oc exec mysql-1-abc123 -- \
  mysqldump -u $MYSQL_USER -p$MYSQL_PASSWORD $MYSQL_DATABASE \
  > backup-$(date +%Y%m%d).sql
```

### MySQL Restore

```shell
oc exec -i mysql-1-abc123 -- \
  mysql -u $MYSQL_USER -p$MYSQL_PASSWORD $MYSQL_DATABASE \
  < backup-20250115.sql
```

---

## Backing Up Persistent Volume Data

### Copy Files from a PVC

```shell
# Copy a directory from a pod to your local machine
oc rsync <pod-name>:/path/to/data ./local-backup/

# Example: back up an uploads directory
oc rsync my-app-1-abc123:/opt/app-root/uploads ./uploads-backup/

# Copy a specific file
oc cp my-app-1-abc123:/opt/app-root/data/important.db ./important.db
```

### Restore Files to a PVC

```shell
# Copy files back to the pod
oc rsync ./local-backup/ <pod-name>:/path/to/data/

# Copy a specific file
oc cp ./important.db my-app-1-abc123:/opt/app-root/data/important.db
```

---

## Automated Backup with CronJobs

### PostgreSQL Daily Backup CronJob

```yaml
apiVersion: batch/v1
kind: CronJob
metadata:
  name: postgres-backup
spec:
  schedule: "0 2 * * *"  # Daily at 2 AM
  concurrencyPolicy: Forbid
  successfulJobsHistoryLimit: 7
  failedJobsHistoryLimit: 3
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: backup
            image: registry.redhat.io/rhel8/postgresql-13
            command:
            - /bin/bash
            - -c
            - |
              TIMESTAMP=$(date +%Y%m%d-%H%M%S)
              pg_dump -h $DB_HOST -U $DB_USER $DB_NAME \
                > /backups/backup-$TIMESTAMP.sql
              find /backups -name "*.sql" -mtime +7 -delete
              echo "Backup completed: backup-$TIMESTAMP.sql"
            env:
            - name: DB_HOST
              value: postgresql
            - name: DB_USER
              valueFrom:
                secretKeyRef:
                  name: postgresql
                  key: database-user
            - name: DB_NAME
              valueFrom:
                secretKeyRef:
                  name: postgresql
                  key: database-name
            - name: PGPASSWORD
              valueFrom:
                secretKeyRef:
                  name: postgresql
                  key: database-password
            volumeMounts:
            - name: backup-storage
              mountPath: /backups
          restartPolicy: OnFailure
          volumes:
          - name: backup-storage
            persistentVolumeClaim:
              claimName: backup-pvc
```

```shell
# Apply the CronJob
oc apply -f postgres-backup-cronjob.yaml

# Check CronJob status
oc get cronjobs

# Trigger a manual backup now
oc create job manual-backup --from=cronjob/postgres-backup
```

---

## Disaster Recovery Scenarios

### Scenario 1: Accidentally Deleted a Deployment

```shell
# If you exported YAML earlier
oc apply -f my-app-deployment.yaml

# If you still have the image in the registry
oc new-app --name=my-app --image=<image-url>

# Rebuild from source
oc new-app https://bitbucket.org/your-org/your-app.git --name=my-app
```

### Scenario 2: Database Corruption

```shell
# 1. Scale down the app to prevent further writes
oc scale deployment/my-app --replicas=0

# 2. Restore from the latest backup
oc exec -i postgresql-1-abc123 -- \
  psql -U $POSTGRESQL_USER $POSTGRESQL_DATABASE < backup-latest.sql

# 3. Scale the app back up
oc scale deployment/my-app --replicas=2
```

### Scenario 3: Entire Project Needs Rebuild

```shell
# 1. Create the project
oc new-project my-project-restored

# 2. Apply all saved resources
oc apply -f my-project-backup.yaml

# 3. Restore database from backup
oc exec -i postgresql-1-abc123 -- psql ... < backup.sql

# 4. Restore PVC data
oc rsync ./local-backup/ my-app-pod:/opt/app-root/data/
```

### Scenario 4: Accidentally Deleted a Secret

```shell
# If you exported it
oc apply -f my-db-credentials.yaml

# If you need to recreate it
oc create secret generic my-db-credentials \
  --from-literal=database-user=myuser \
  --from-literal=database-password=mypassword \
  --from-literal=database-name=mydb
```

---

## Backup Checklist

| Item | Status | Command |
| --- | --- | --- |
| Source code in Git | Required | `git status` |
| Resource YAML exported | Weekly | `oc get all -o yaml > backup.yaml` |
| Database backup scheduled | Daily | `oc get cronjobs` |
| PVC data backed up | As needed | `oc rsync pod:/data ./backup/` |
| Secrets documented (not in Git) | After changes | `oc get secrets` |
| Backup restore tested | Monthly | Restore to a test project |

---

## Best Practices

| Practice | Why |
| --- | --- |
| **Store backups off-cluster** | If the cluster goes down, backups survive |
| **Test restores regularly** | A backup you can't restore is worthless |
| **Automate with CronJobs** | Manual backups get forgotten |
| **Version your YAML in Git** | Track changes to infrastructure config |
| **Keep 7+ days of backups** | Gives time to notice issues |
| **Document your recovery steps** | When things break, you won't have time to figure it out |
| **Encrypt sensitive backups** | Database dumps may contain credentials |

---

## Quick Reference

```shell
# Export resources
oc get all,configmap,secret,pvc,route -o yaml > backup.yaml

# Database backup
oc exec <db-pod> -- pg_dump -U $USER $DB > backup.sql
oc exec <db-pod> -- mysqldump -u $USER -p$PASS $DB > backup.sql

# Database restore
oc exec -i <db-pod> -- psql -U $USER $DB < backup.sql
oc exec -i <db-pod> -- mysql -u $USER -p$PASS $DB < backup.sql

# File backup/restore
oc rsync <pod>:/data ./backup/       # Backup
oc rsync ./backup/ <pod>:/data/      # Restore

# CronJob management
oc get cronjobs                       # List scheduled backups
oc create job manual --from=cronjob/postgres-backup  # Run now
```

---

*UConn ITS Infrastructure Services — OpenShift Knowledge Base*