Task 2 — Rollback Requirement

Rollback Demonstration

Complete rollback capability including documented method, backup/restore methodology, and migration safety protocol.

Backup Method

Full MariaDB dump + file system backup using bench CLI. Backups are compressed (gzip) and checksummed (SHA-256) for integrity verification.

Recovery Time

Estimated rollback time: 15-30 minutes depending on database size. Tested on staging with full dataset. RTO target: under 1 hour.

Safety Protocol

Pre-migration backup is mandatory. No migration proceeds without verified backup. Staging environment tested first. Production migration during maintenance window only.

Interactive Rollback Simulation

Step 1Create Pre-Migration Backup

Full database and files backup before any migration begins

$ bench --site erp.shangu.co.za backup --with-files

Step 2Verify Backup Integrity

Validate backup file checksums and test restore on staging

$ bench --site erp.shangu.co.za verify-backup --backup-path ./backups/latest

Step 3Stop All Services

Gracefully stop web server and background workers

$ sudo supervisorctl stop all

Step 4Restore Database

Restore MariaDB database from pre-migration backup

$ bench --site erp.shangu.co.za restore ./backups/pre-migration.sql.gz

Step 5Restore Files

Restore uploaded files and private files from backup

$ bench --site erp.shangu.co.za restore --with-files ./backups/files.tar.gz

Step 6Clear Cache

Clear Redis cache and rebuild search index

$ bench --site erp.shangu.co.za clear-cache && bench --site erp.shangu.co.za clear-website-cache

Step 7Run Migrations

Ensure database schema matches the pre-migration state

$ bench --site erp.shangu.co.za migrate

Step 8Restart Services

Start all services and verify system health

$ sudo supervisorctl start all && bench --site erp.shangu.co.za doctor

Step 9Post-Rollback Verification

Run reconciliation queries to confirm data matches pre-migration state

$ bench --site erp.shangu.co.za execute shangu_erp.utils.verify_rollback

Migration Safety Protocol

Pre-Migration Checklist

Full database backup created and verified

File system backup (public + private files)

Backup checksums recorded and validated

Staging environment tested with full rollback

Maintenance window scheduled and communicated

Rollback procedure documented and reviewed

Team members assigned rollback responsibilities

Monitoring alerts configured for migration

Rollback Triggers

Trial balance does not reconcile after import

More than 0.5% of transactions fail validation

Referential integrity errors detected

Stock valuation variance exceeds acceptable threshold

System performance degrades beyond acceptable limits

Critical business process blocked by imported data

Data corruption detected in any DocType

Stakeholder requests rollback during review period

Backup & Restore Methodology

# ═══════════════════════════════════════════════════════
# SHANGU GROUP - Migration Backup & Restore Procedure
# ═══════════════════════════════════════════════════════

# 1. PRE-MIGRATION BACKUP
bench --site erp.shangu.co.za backup --with-files
# Output: Database SQL dump + files tarball in ./sites/erp.shangu.co.za/private/backups/

# 2. VERIFY BACKUP INTEGRITY
sha256sum ./backups/*.sql.gz > ./backups/checksums.txt
bench --site staging.shangu.co.za restore ./backups/latest.sql.gz --with-files
# Verify staging site loads correctly

# 3. MIGRATION EXECUTION
bench --site erp.shangu.co.za execute shangu_erp.migration.run_full_migration
# Monitor: bench --site erp.shangu.co.za execute shangu_erp.migration.get_status

# 4. POST-MIGRATION VERIFICATION
bench --site erp.shangu.co.za execute shangu_erp.reconciliation.run_all_checks
# If any check fails → trigger rollback

# 5. ROLLBACK (if needed)
sudo supervisorctl stop all
bench --site erp.shangu.co.za restore ./backups/pre-migration.sql.gz --with-files
bench --site erp.shangu.co.za clear-cache
bench --site erp.shangu.co.za migrate
sudo supervisorctl start all
bench --site erp.shangu.co.za execute shangu_erp.utils.verify_rollback

# 6. POST-ROLLBACK VERIFICATION
# Confirm all data matches pre-migration state
# Confirm all business processes operational

Data Corruption Prevention: The bench restore command performs an atomic database restore — either the entire backup is restored successfully, or the operation fails completely without partial data. This prevents data corruption during rollback scenarios.