Before you begin
Verify your AlloyDB Omni Kubernetes operator version is
1.6.3. To upgrade your AlloyDB Omni operator version, see Migrate to the latest AlloyDB Omni Kubernetes operator version.Verify you have at least 5GB of free disk space.
If you run external or remote backup systems, perform a full backup and pause any scheduled backups to avoid backing up during the upgrade process.
For high availability (HA) and data resilience (DR) environments, decouple your database clusters before upgrading them.
Upgrade your AlloyDB Omni version
To upgrade your AlloyDB Omni version using pg_upgrade, do one
of the following:
Run the Bash script that performs the upgrade for you.
Perform scripted upgrade
You can find a script that performs the manual upgrade steps in the AlloyDB Omni samples on GitHub.
To use the upgrade script, download the script and run the following command:
./MVU.sh 17.5.0 1.6.3 NAME NAMESPACE
Replace the following variables:
- NAME: the name of your database cluster.
- NAMESPACE: the namespace of your database cluster.
Perform manual upgrade steps
Follow the instructions for the major version you upgrade from.
Upgrade from an earlier major version
Get the current and target major versions of your database.
export OLD_PG_VERSION=$(kubectl get dbcluster.alloydbomni.dbadmin.goog NAME -n NAMESPACE -o jsonpath='{.spec.databaseVersion}' | cut -d'.' -f1) export NEW_PG_VERSION=$(echo '17.5.0' | cut -d'.' -f1)Replace the following variables:
NAME: the name of your database cluster.NAMESPACE: the namespace for your database cluster.
Back up your data to avoid any data loss.
Terminate all replication processes. This includes physical replication used for high availability (HA) and data resilience (DR) as well as all logical replication streams.
Annotate your database cluster to specify a manual version upgrade.
kubectl annotate dbclusters.alloydbomni.dbadmin.goog NAME -n NAMESPACE dbs.dbadmin.goog.com/manualmvu=trueReplace the following variables:
NAME: the name of your database cluster.NAMESPACE: the namespace for your database cluster.
Stop the PostgreSQL process and move the data for the upgrade.
kubectl exec -i POD_NAME -n NAMESPACE -c database -- /bin/bash -c " # Shut down postgres supervisorctl.par stop postgres; # Check data directory and move it for upgrade if necessary if [ -d /mnt/disks/pgsql/data ]; then echo 'Found data directory at /mnt/disks/pgsql/data, moving it to /mnt/disks/pgsql/${OLD_PG_VERSION}/db' mkdir -p /mnt/disks/pgsql/${OLD_PG_VERSION}; mv /mnt/disks/pgsql/data /mnt/disks/pgsql/${OLD_PG_VERSION}/db; elif [ ! -d /mnt/disks/pgsql/${OLD_PG_VERSION}/db ]; then echo 'ERROR: Neither /mnt/disks/pgsql/data nor /mnt/disks/pgsql/${OLD_PG_VERSION}/db directory found' >&2; exit 1; fi cp -r /usr/lib/postgresql/${OLD_PG_VERSION}/bin /mnt/disks/pgsql/${OLD_PG_VERSION}/.; cp -r /usr/lib/postgresql/${OLD_PG_VERSION}/lib /mnt/disks/pgsql/${OLD_PG_VERSION}/.; cp -r /usr/share/postgresql/${OLD_PG_VERSION} /mnt/disks/pgsql/${OLD_PG_VERSION}/share; rm -f /mnt/disks/pgsql/${OLD_PG_VERSION}/share/postgresql.conf.sample; cp /usr/share/postgresql/postgresql.conf.sample /mnt/disks/pgsql/${OLD_PG_VERSION}/share/postgresql.conf.sample; chmod 2740 /mnt/disks/pgsql/${OLD_PG_VERSION}/db "Replace the following variables:
POD_NAME: the name of your pod.NAMESPACE: the namespace for your database cluster.
Patch your database cluster to version
17.5.0and your control plane agents (CPA) to1.6.3.kubectl patch dbclusters.alloydbomni.dbadmin.goog NAME -n NAMESPACE --type=merge -p '{"spec":{"databaseVersion":"17.5.0","controlPlaneAgentsVersion":"1.6.3","databaseImageOSType":"UBI9"}}'Replace the following variables:
NAME: the name of your database cluster.NAMESPACE: the namespace for your database cluster.
Get the post-upgrade data directory path and set it as an environment variable.
export POST_UPGRADE_DATA_DIR=$(kubectl exec -i POD_NAME -n NAMESPACE -c database -- /bin/bash -c "if [ -d /mnt/disks/pgsql/data ]; then echo /mnt/disks/pgsql/data; else echo /mnt/disks/pgsql/${NEW_PG_VERSION}/db; fi")Replace the following variables:
POD_NAME: the name of your pod.NAMESPACE: the namespace for your database cluster.
Create an empty database cluster for the upgraded AlloyDB Omni version.
kubectl exec -i POD_NAME -n NAMESPACE -c database -- /bin/bash -c " supervisorctl.par stop postgres; # Initialize a new, empty database cluster for the new version. rm -fr $POST_UPGRADE_DATA_DIR; initdb -D $POST_UPGRADE_DATA_DIR -U alloydbadmin --data-checksums --encoding=UTF8 --locale=C --locale-provider=icu --icu-locale=und-x-icu --auth-host=trust --auth-local=reject; cd ~ "Replace the following variables:
POD_NAME: the name of your pod.NAMESPACE: the namespace for your database cluster.
Modify
pg_hba.confin your pre-existing database cluster to provide the upgrade process the permissions and connections it needs.kubectl exec -i POD_NAME -n NAMESPACE -c database -- /bin/bash -c " # Backup pg_hba.conf of the old postgres cluster cp /mnt/disks/pgsql/${OLD_PG_VERSION}/db/pg_hba.conf /mnt/disks/pgsql/${OLD_PG_VERSION}/db/pg_hba.conf.bak; # Update old postgres configurations echo 'local all all trust' >> /mnt/disks/pgsql/${OLD_PG_VERSION}/db/pg_hba.conf; echo 'host all all 127.0.0.1/32 trust' >> /mnt/disks/pgsql/${OLD_PG_VERSION}/db/pg_hba.conf "Replace the following variables:
POD_NAME: the name of your pod.NAMESPACE: the namespace for your database cluster.
Modify
pg_hba.confin the new database cluster to provide the upgrade process the permissions and connections it needs.kubectl exec -i POD_NAME -n NAMESPACE -c database -- /bin/bash -c " rm $POST_UPGRADE_DATA_DIR/pg_hba.conf; echo 'local all all trust' >> $POST_UPGRADE_DATA_DIR/pg_hba.conf; echo 'host all all 127.0.0.1/32 trust' >> $POST_UPGRADE_DATA_DIR/pg_hba.conf; "Replace the following variables:
POD_NAME: the name of your pod.NAMESPACE: the namespace for your database cluster.
Check for upgrade eligibility.
kubectl exec -i POD_NAME -n NAMESPACE -c database -- /bin/bash -c " cd /mnt/disks/pgsql; # Make sure postgres is stopped supervisorctl.par stop postgres; # Check to verify that data can be upgraded. pg_upgrade -U alloydbadmin --check -b /mnt/disks/pgsql/${OLD_PG_VERSION}/bin -B /usr/lib/postgresql/${NEW_PG_VERSION}/bin -d /mnt/disks/pgsql/${OLD_PG_VERSION}/db -D $POST_UPGRADE_DATA_DIR --link -v; "Replace the following variables:
POD_NAME: the name of your pod.NAMESPACE: the namespace for your database cluster.
Perform the database upgrade.
kubectl exec -i POD_NAME -n NAMESPACE -c database -- /bin/bash -c " cd /mnt/disks/pgsql; pg_upgrade -U alloydbadmin -b /mnt/disks/pgsql/${OLD_PG_VERSION}/bin -B /usr/lib/postgresql/${NEW_PG_VERSION}/bin -d /mnt/disks/pgsql/${OLD_PG_VERSION}/db -D $POST_UPGRADE_DATA_DIR --link -v; # Recover old cluster configurations cp /mnt/disks/pgsql/${OLD_PG_VERSION}/db/pg_hba.conf.bak $POST_UPGRADE_DATA_DIR/pg_hba.conf; cp -r /mnt/disks/pgsql/${OLD_PG_VERSION}/db/postgresql.conf $POST_UPGRADE_DATA_DIR/.; cp -r /mnt/disks/pgsql/${OLD_PG_VERSION}/db/postgresql.conf.d $POST_UPGRADE_DATA_DIR/.; cp -r /mnt/disks/pgsql/${OLD_PG_VERSION}/db/parambackup $POST_UPGRADE_DATA_DIR/.; # Start postgres supervisorctl.par start postgres; # Clean up backup data rm -fr /mnt/disks/pgsql/${OLD_PG_VERSION}/ "Replace the following variables:
POD_NAME: the name of your pod.NAMESPACE: the namespace for your database cluster.
After a successful upgrade, remove the manual version upgrade annotation from your database cluster.
kubectl annotate dbclusters.alloydbomni.dbadmin.goog NAME -n NAMESPACE dbs.dbadmin.goog.com/manualmvu-Replace the following variables:
NAME: the name of your database cluster.NAMESPACE: the namespace for your database cluster.
Resume normal operations and re-enable any backups, backup plans, or replication processes that you disabled in steps one through three.