8.3 KiB
| date | title | tags | |||
|---|---|---|---|---|---|
| 2024-03-28 | Migrating Nextcloud from Local Storage to S3 |
|
As I continue to investigate migration from Unraid to XCP-ng (or maybe Proxmox), I am taking a closer look at what storage resources my containers are still using on my Unraid system. Unsurprisingly, Nextcloud is using a significant amount of persistent storage and I found that moving that data to a remote share is difficult due to Nextcloud's file permission requirements. Looking for a solution, I found that Nextcloud supports S3 storage and, while not officially supported, there are ways to move from local storage to S3 without having to start over from scratch. I will admit that much of this was heacily inspired by other guides which I've link below, but there were a few gotchas related to doing this with containers in Unraid that I'll highlight.
Why to migrate storage
As I mentioned, my primary reason for migrating from local to S3 storage is to simplify running Nextcloud in a container with primary storage residing on a different system. This is also a fun project for me to learn a little more about how S3 works (this is my first application using S3 storage 🎉). I'm using MinIO on TrueNAS which was practically a one-click deployment; I followed this video from Lawrence Systems.
Preparing for migration
Fist things first, configure a bucket for Nextcloud (and only Nextcloud) to use; I've named mine "nextcloud".
I only have a few users on my Nextcloud instance and I'm the only one who uses it regularly, so I was able to
prevent any changes during the migration by closing any desktop applications that sync and not taking any
photos on my phone. If you have more users or activity, I would recommend putting Nextcloud into maintenance
mode before migrating things (occ maintenance:mode --on).
Migrating from local to S3
Other guides I found made some assumptions about Nextcloud and SQL running in the same environment, which
isn't true in my case where they're in containers. They also assumed the aws cli could be used where
Nextcloud is running which is also not true in my case since Unraid doesn't have that package available.
The code snippets below have been modified from other examples to fit my use case.
Get files from SQL
My Nextcloud setup uses MariaDb, so the first step is to export the list of files, along with some identifiers
Nextcloud uses, to be uploaded. My database is called nextcloud and the /config directory is mounted to the
host. /mnt/user/Nextcloud on the host system is mounted to /data in the Nextcloud container.
mysql -p -B --disable-column-names -D nextcloud << EOF > /config/meta_file_list
select concat('urn:oid:', fileid, ' ', substring(id from 8), path)
from oc_filecache
join oc_storages
on storage = numeric_id
where id like 'local::%'
order by id;
EOF
mysql -p -B --disable-column-names -D nextcloud << EOF > /config/user_file_list
select concat('urn:oid:', fileid, ' ', '/mnt/user/Nextcloud/',
substring(id from 7), '/', path)
from oc_filecache
join oc_storages
on storage = numeric_id
where id like 'home::%'
order by id;
EOF
sed -i -e "s| /data/| /mnt/user/Nextcloud/|g" meta_file_list
Note that user_file_list paths are updated in the mysql command and the meta_file_list is updated
by the sed command to be accessible on the host.
Move files to upload directory
Since I will be mounting the upload directory to a virtual machine, I opted to copy files rather than
create symlinks like the example I used as reference. It doesn't particularly matter where on the host
system the files to upload are copied to, but I used /mnt/user/Nextcloud which was my existing share
for Nextcloud data. I copied user_file_list and meta_file_list from the previous step to that path.
cd /mnt/user/Nextcloud
mkdir s3_files
cd s3_files
while read target source ; do
if [ -f "$source" ] ;
then
cp "$source" "$target"
fi
done < ../user_file_list
while read target source ; do
if [ -f "$source" ] ;
then
cp "$source" "$target"
fi
done < ../meta_file_list
At this point, all of the files are named and placed in the s3_files directory, ready to be uploaded
to S3. Note that since the filenames contain :, this directory will NOT be accessible via an SMB share.
I exported it as an NFS share, but if I could install awscli on the host system then it wouldn't have
been a concern.
On my VM where I actually uploaded everything to a bucket named nextcloud:
sudo mount <SERVER_ADDR>:/mnt/user/Nextcloud /home/$USER/Nextcloud
cd ~/Nextcloud
aws --endpoint-url <S3_URL> s3 sync s3_files s3://nextcloud
Nextcloud config updates
With all of the files uploaded to S3, Nextcloud needs to be updated to point at the new data source. So, time to put Nextcloud into maintenance mode and start breaking things 🙂. From here on out, we are making potentially destructive changes, so make backups, have a recovery plan, and don't continue until reading to the end first.
In the Docker container:
occ maintenance:mode --on
nano /config/www/nextcloud/config/config.php
Add the following to the Nextcloud config (/config/www/nextcloud/config/config.php):
'objectstore' =>
array (
'class' => 'OC\\Files\\ObjectStore\\S3',
'arguments' =>
array (
'bucket' => 'nextcloud',
'autocreate' => true,
'key' => '<MY_AWS_KEY>',
'secret' => '<MY_AWS_SECRET>',
'hostname' => '<S3_ADDRESS>',
'port' => 443,
'use_ssl' => true,
'region' => 'home',
'use_path_style' => true,
),
),
Make sure this is before the closing
);at the end of the file
use_path_style was required for MinIO, but I read that this may be false for other providers.
SQL database updates
The SQL database at this point still contains local file references, so those will need to be updated to the uploaded resources. In the SQL container:
mariadb -p -D nextcloud -e "update oc_storages set id = concat('object::user:', substring(id from 7)) where id like 'home::%';"
mariadb -p -D nextcloud -e "update oc_storages set id = 'object::store:amazon::nextcloud' where id like 'local::%';"
mariadb -p -D nextcloud -e "update oc_mounts set mount_provider_class = 'OC\\\Files\\\Mount\\\ObjectHomeMountProvider' where mount_provider_class like '%LocalHomeMountProvider%';"
Validating Changes
With configuration and database changes completed, its time to take Nextcloud out of maintenance mode and validate changes. In the Nextcloud container:
occ maintenance:mode --off
I also restarted the Nextcloud container here for good measure, but it may not be necessary.
Load up Nextcloud in a browser and make sure the page loads, user login works, and most importantly user files are accessible. If this all works, then the migration is complete!
Cleanup local files
With migration completed, I chose to remove the data container mount since the directory only needs
the .ocdata file and should only contain logs. On the host system:
cp /mnt/user/Nextcloud/.oc_data /mnt/user/appdata/nextcloud/data/
chmod 770 /mnt/user/appdata/nextcloud/data/
chown nobody:users /mnt/user/appdata/nextcloud/data/
Note that the ownership is specific to Unraid, but Nextcloud requires permissions
770.
Then, modify Nextcloud's config.php:
'datadirectory' => '/config/data',
Then the /data mount can be removed from the Docker container and the container restarted. At this point,
the old data share can be removed completely (I'll archive mine as a backup).
Wrap up
I have been using my Nextcloud with S3 storage for a few days now and haven't noticed any difference compared to local storage. I like that the storage is accessible via HTTP so I don't have to worry about managing permissions and access from the Nextcloud container to my storage server.