Store everything on a fault-tolerant disk array (RAID 1 or 5 or better).

Use battery backup.

Use more reliable hardware, such as SCSI instead of IDE.

Store backups on a third disk on another controller

Store backups on a different computer on a different network in a different physical location. (Compared to off-line backup such as tapes and CDRs, on-line backup is faster and more likely to succeed, but requires maintenance of another machine.)

Plan and configure for recovery from the beginning.

Test your recovery strategy from time to time.

Make it easy to maintain and test your recovery strategy, so that you are more likely to do it.

PostGreSQL. Create a backup file and verify that it was created and has a reasonable size (several megabytes).

[root@localhost root]# su - service0
[service0@localhost service0]$ pg_dump -f /var/lib/aolserver/service0/database-backup/before_upgrade_to_4.6.dmp service0
[service0@localhost service0]$ ls -al /var/lib/aolserver/service0/database-backup/before_upgrade_to_4.6.dmp 
-rw-rw-r-x    1 service0  service0   4005995 Feb 21 18:28 /var/lib/aolserver/service0/database-backup/before_upgrade_to_4.6.dmp
[service0@localhost service0]$ exit
[root@localhost root]#
su - service0
pg_dump -f /var/lib/aolserver/service0/database-backup/before_upgrade_to_4.6.dmp openacs-dev
ls -al /var/lib/aolserver/service0/database-backup/before_upgrade_to_4.6.dmp
exit

File tree with CVS. If you are already using CVS, you probably don't need to do anything to back up your data. Just make sure that your current work is checked into the system. You can then roll back based on date - note the current system time, down to the minute. For maximum safety, you can apply a tag to your current files. Note that, if you did the CVS options in this document, the /var/lib/aolserver/service0/etc directory is not included in cvs and you may want to add it.

[root@localhost root]# su - service0
[service0@localhost service0]$ cd /var/lib/aolserver/service0
[service0@localhost service0]$ cvs commit -m "last-minute commits before upgrade to 4.6"
cvs commit: Examining .
cvs commit: Examining bin
(many lines omitted)
[service0@localhost service0]$ cvs tag before_upgrade_to_4_6
cvs server: Tagging bin
T bin/acs-4-0-publish.sh
T bin/ad-context-server.pl
(many lines omitted)
[service0@localhost service0]$ exit
[root@localhost root]# 
su - service0
cd /var/lib/aolserver/service0
cvs commit -m "last-minute commits before upgrade to 4.6"
cvs tag before_upgrade_to_4_6
exit

To restore files from a cvs tag such as the one used above:

[root@localhost root]# su - service0
[service0@localhost service0]$ cd /var/lib/aolserver/service0
[service0@localhost service0]$ cvs up -r current
[service0@localhost service0]$ exit
su - service0
cd /var/lib/aolserver/service0
cvs up -r current

File tree without CVS. If you don't use cvs, you may want to back up the working directory. The simplest way is just to copy it.

[root@localhost root]# su - service0
[service0@localhost service0]$ cp -r /var/lib/aolserver/service0 /var/lib/aolserver/service0-before-upgrade-to-4.6
[service0@localhost service0]$ exit
[root@localhost root]# 
su - service0
cp -r /var/lib/aolserver/service0 /var/lib/aolserver/service0-before-upgrade-to-4.6
exit

To restore the files, copy the directory back (do this while the service is stopped):

[root@localhost root]# su - service0
[service0@localhost service0]$ mv /var/lib/aolserver/service0 /var/lib/aolserver/openacs-failed-upgrade
[service0@localhost service0]$ mv /var/lib/aolserver/service0-before-upgrade-to-4.6 /var/lib/aolserver/service0
[service0@localhost service0]$ exit
[root@localhost root]#

Database Rollback. Restore the database to its backed-up state.

[root@localhost root]# su - service0
[service0@localhost service0]$ svc -d /service/service0
[service0@localhost service0]$ dropdb service0
DROP DATABASE
[service0@localhost service0]$ createdb service0
CREATE DATABASE

PostGreSQL's dump command does not guarantee to back up all of the procedures and things in the right order for them to be reassembled. In practice, OpenACS users have found that rebuilding some of the common procedures before running the restore usually addresses this. You will see a number of "already exists" errors when you run the database restore; these can be ignored. This forum thread has more information.

[service0@localhost service0]$ psql -f /var/lib/aolserver/service0/packages/acs-kernel/sql/postgresql/postgresql.sql service0
[service0@localhost service0]$ psql service0 < /backup/openacs/openacs_dev_before_upgrade_to_4.6.dmp
[service0@localhost service0]$ svc -u /service/service0
[service0@localhost service0]$ exit

All commands for a database rollback:
su - service0
svc -d /service/service0
dropdb service0
createdb service0
psql -f /var/lib/aolserver/service0/packages/acs-kernel/sql/postgresql/postgresql.sql service0
psql service0 < /var/lib/aolserver/service0/database-backup/before_upgrade_to_4.6.dmp
svc -u /service/service0
exit

c create a new tar archive

p preserves permissions.

s preserves file sort order

j compresses the output with bz2.

The --exclude clauses skips some daemontools files that are owned by root and thus cannot be backed up by the service owner. These files are autogenerated and we don't break anything by omitting them.

The --file clause specifies the name of the output file to be generated; we manually add the correct extensions.

The last clause, /var/lib/aolserver/service0/, specifies the starting point for backup. Tar defaults to recursive backup.