We are about to upgrade one of our GlusterFS-based storage systems at work. Fortunately, I was worrying about the upgrade procedure for the Debian packages not being tested by the maintainers. It turns out I was right. Simply upgrading the packages without manual intervention will apparently render your GlusterFS server unusable.
I have only tested the most basic distributed GlusterFS setup. No replication whatsoever. We have two GlusterFS servers, storage1 and storage2. A peering between both has been established, and a very basic volume has been configured:
storage1:~# gluster gluster> peer status Number of Peers: 1 Hostname: storage2 Uuid: 2d22cc13-2252-4cf1-bfe9-3d27fa2fbc29 State: Peer in Cluster (Connected) gluster> volume create data storage1:/srv/data storage2:/srv/data ... gluster> volume start data ... gluster> volume info Volume Name: data Type: Distribute Volume ID: e2bd5767-4b33-4e57-9320-91ca76f52d56 Status: Started Number of Bricks: 2 Transport-type: tcp Bricks: Brick1: storage1:/srv/data Brick2: storage2:/srv/data
For the test setup, I populated the volume with a number of files.
Upgrading from Wheezy to Jessie
To be save, stop the volume before you begin with the package upgrade:
gluster> volume stop data
And now perform your dist-upgrade.
After the upgrade, you will have to perform two manual clean ups. Both actions have to be performed on all storage servers.
/etc/glusterd is now /var/lib/glusterd
The package maintainers have apparently neglected to take care of this one. You manually need to copy the old configuration files over.
storage1:~# cd /var/lib/glusterd && cp -r /etc/glusterd/* .
Put volume-id in extended attribute
GlusterFS 3.5 requires the volume-id in an extended directory attribute. This is also not automatically handled during package upgrade.
storage1:~# vol=data storage1:~# volid=$(grep volume-id /var/lib/glusterd/vols/$vol/info | cut -d= -f2 | sed 's/-//g') storage1:~# setfattr -n trusted.glusterfs.volume-id -v 0x$volid /srv/data
With these two steps performed on all GlusterFS servers, you should now be able to start and mount your volume again in Debian Jessie.
Do not forget to explicitly stop the volume again before continueing with the next upgrade step.
Upgrading from Jessie to Stretch
After you have dist-upgraded to Stretch, there is yet another manual step you have to take to convert the volume metadata to the new layout in GlusterFS 3.8. Make sure you have stopped your volumes and the GlusterFS server.
storage1:~# service glusterfs-server stop
Now run the following command:
storage1:~# glusterd --xlator-option *.upgrade=on -N
Now you should be ready to start your volume again:
storage1:~# service glusterfs-server start storage1:~# gluster gluster> volume start data
And mount it:
client:~# mount -t glusterfs storage1:/data /mnt
You should now be running GlusterFS 3.8 and your files should still all be there.