forgejo/git/repositories/d_mcknight/blog-content.git/pulls/2.patch

From 10530c133100eafb475d51c7171f4bb51d114f7d Mon Sep 17 00:00:00 2001
From: Daniel McKnight <daniel@mcknight.tech>
Date: Fri, 19 Jan 2024 06:53:36 +0000
Subject: [PATCH 1/2] Update file 2023-01-18_NAS_Setup.md

---
 2023-01-18_NAS_Setup.md | 155 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 155 insertions(+)
 create mode 100644 2023-01-18_NAS_Setup.md

diff --git a/2023-01-18_NAS_Setup.md b/2023-01-18_NAS_Setup.md
new file mode 100644
index 0000000..34abe1d
--- /dev/null
+++ b/2023-01-18_NAS_Setup.md
@@ -0,0 +1,155 @@
+---
+date: 2024-01-18
+title: Configuring TrueNAS
+tags: 
+  - homelab
+  - truenas
+  - storage
+  - backup
+---
+I [previously](https://blog.mcknight.tech/2024/01/11/Storage-Server/) talked about my physical storage server
+build and choice of TrueNAS SCALE as an operating system. I did configure a single storage pool and copy my 
+data over since then to experiment with settings and see how things work. I had enough spare drives so I was
+able to do this without touching my Unraid server which made for an easier and less stressful transition.
+I'll go over my storage architecture plan, accounting for what I learned in this initial setup.
+
+## Storage Pools
+I plan on creating two pools, one pool of large HDDs with a Metadata VDEV for general file storage and another
+pool of SSDs for application data and VMs.
+
+Initially, I started with a pool of HDDs named "Rusty" and have been working out the details for share
+settings and permissions. My current containers and VMs in Unraid are still using the SSD cache on that
+box for application data, but I've moved all other data to my new NAS and have applications like Plex and
+Nextcloud pointed at the new TrueNAS shares. My plan is to add a pool of SSDs and move my application data
+and VM disks there before re-purposing my Unraid server.
+
+### VDEV Configuration
+I have my primary storage pool configured with a 5 disk RAIDZ2 Data VDEV and a 2 disk Mirrored Metadata VDEV.
+This means I have data on an array that will tolerate two drive failures and the file metadata on an array
+that will tolerate one drive failure. I'm fairly comfortable with this configuration since the metadata
+device is using SSDs that should be less prone to failure and that should rebuild more quickly in the event of
+a failure.
+
+My plan is to change this to a RAIDZ configuration with the same 2 cache disks. I already have a comprehensive
+backup plan (more on that below), so I'm comfortable with the risk of only tolerating one drive failure. I also 
+intend on adding another drive or two to my primary storage pool to keep my primary and backup arrays about
+the same size.
+
+## Datasets
+Without going into too much detail, a dataset can be thought of as a directory in a ZFS pool. Each dataset
+can have different permissions, encryption, and compression settings applied and datasets may be nested. In
+addition to datasets, Zvols can be created to provision block storage for iSCSI targets (more on that later). 
+
+### "Rusty" Dataset Configuration
+I created the root dataset as unencryptedt with LZ4 Compression and no de-duplication. Under that, I 
+re-created my Unraid shares as datasets (except for the "Games" share which I configured as a Zvol for an
+iSCSI share for better compatibility). I'll work through each of my datasets to cover some of my configuration
+choices and why I chose them; not everyone will structure data in the same way, but I believe this covers most
+common use cases for network shares.
+
+#### Backup
+This share contains manual backups and archived data, for example I will backup user directories before
+recycling or repurposing an old computer. I will also take a manual backup of my pictures before trying 
+new software like Lightroom or Darktable that has the ability to remove files it manages. This share also
+includes files that I might otherwise just delete, like a local Nextcloud directory with sync conflicts or
+old configuration/application data backups I made before making changes.
+
+I have this set to use ZSTD-19 Compression (highest compression at the expense of CPU usage) with ZFS
+Deduplication enabled and Key-based encryption. Deduplication on this share is VERY beneficial for me; for
+example, I have multiple backups of my photos and Lightroom library which will always overlap previous backups.
+Arguably, I should just clear out old backups when I create new ones, but with deduplication it doesn't cost 
+anything to keep the duplicated backup.
+
+#### Games
+This is an unencrypted Zvol that I can mount to a computer and treat like a local hard drive. This is useful
+when you need to expose block storage to a client and not a network share, i.e. program installation. 
+This volume is where I install games to and also keep save data. I followed a tutorial from Craft Computing 
+[on youtube](https://www.youtube.com/watch?v=9JL-RVUHj6o) which shows the exact process and also provides
+more information about Zvols.
+
+#### Installers
+I keep driver installers, purchased software, and OS images here. Basic LZ4 compression, deduplication, and no
+encryption since nothing here is sensitive. I don't know that deduplication is really necessary here, but it
+shouldn't cost much performance and this share is primarily read from and not often written.
+
+#### Media
+Media libraries; unencrypted with no compression and no deduplication. Most media will not compress much if at
+all, so I think it would be a waste of CPU cycles to enable it here. I also manage my media libraries and know
+there are no duplicate files.
+
+#### Nextcloud
+Nextcloud user share data, encrypted with LZ4 compression and no de-duplication. I may update this to enable
+de-duplication since user shares might include duplicate files but I didn't see a benefit when setting up the
+share and I only have a few Nextcloud users.
+
+#### Security
+Security camera recordings, encryption with no compression snd no de-duplication. Nothing else to note here.
+
+#### Personal User Share
+Encrypted, compressed, and with de-duplication. This is where I put my pictures and documents, and also the
+configured home directory for my TrueNAS user. I have de-duplication enabled here because I have some software
+projects/downloads duplicated with different names as well as some duplicated pictures due to having multiple
+backups (Plex upload, Nextcloud upload, manual backup before moving to a new phone).
+
+##### Private Dataset
+Same as the personal share its nested under, except this is encrypted with a passphrase. Data in this share
+is generally unaccessible unless I login to the TrueNAS Web UI and unlock it with a password; useful for 
+sensitive data that isn't often accessed.
+
+## Shares
+With all of my data organized into datasets, shares need to be configured so I can access that data from
+client systems. For the most part, this is a 1:1 mapping where each dataset gets an SMB and/or NFS share
+but I will highlight my personal user share since its a little unique.
+
+I have my personal share exposed which includes the Private dataset within it available when unlocked. 
+if the Private dataset is locked, then clients just see an empty directory. 
+This is a nice feature so even authenticated SMB connections won't be
+able to leak sensitive information if I keep the share locked behind a password.
+
+## Data Protection
+I have a couple of backups that run locally with plans to include an offsite backup in the future.
+
+### Manual Backup Target
+I have an old 1U server that I turn on manually every week or 2 to run a replication task. This server has
+1Gig networking and I found that SSH only transferred at around 40Mbps; following some advice in the TrueNAS 
+forums, I enabled SSH + Netcat which got transfers back up to 1Gig pretty consistently. My backup server runs
+Ubuntu server, so I did have to build [libzfs](https://github.com/truenas/py-libzfs) from scratch for this. I
+don't have enough capacity here for a full replication, so I exclude the `Backup` share here.
+
+### Scheduled Backup Target
+I setup a replication task to my Unraid server, where I configured a RAIDZ Pool using the drives previously
+assigned to an Unraid data array. I configured replication using SSH only for this task and I am seeing 
+transfer rates arount 3Gbps which is about the best I would expect from HDDs.
+
+## Users
+I decided to use service accounts for containers to access shares as well as a user acount for myself. I
+created dedicated accounts for Nextcloud, Security, and Media access and used those user groups to grant
+myself access to the relevant shares.
+
+## Other Config
+I didn't do too much manual configuration and mostly left default settings outside of enabling services for
+the various shares. I enabled 2FA to require an authenticator code at login, though I don't intend on exposing
+the admin UI outside my network so this is probably overkill.
+
+I also increase the ARC memory limit after observing fairly constant memory usage by other services. I will 
+note that this seems to be a contentious topic on the forums and some claim that increasing ARC can cause
+system instability. Personally, I know that I'm not running any services that will consume more memory 
+(i.e. containers and VMs) so I feel comfortable allocating much more of my RAM to ARC. I added the below
+startup script to allocate most of my 32GB of RAM to ARC while leaving a small buffer beyond what I see services
+using.
+
+```shell
+echo 30064771072 > /sys/module/zfs/parameters/zfs_arc_max
+```
+
+## Next Up
+As this summary included current and planned configuration, I still have some work to do getting my setup to
+match what I've described. At the moment, I'm replicating my new ZFS dataset to my Unraid server before
+re-creating the pool on my NAS with the changes I discussed here (and with an additional drive in the data
+VDEV).
+
+Once I have my data comfortably in its new home, I will flesh out my backup strategy and be ready to look into
+how I want to structure compute. I mentioned [in my first post](https://blog.mcknight.tech/2023/11/29/Homelab-Upgrades/)
+that I was considering Proxmox as a solution here, but I have since been hearing more about 
+[XCP-ng](https://xcp-ng.org/) and will have to do some more research and probably trial both before making a
+decision.
-- 
GitLab


From 51f92c3137a736adedcbdd282893e87e5a3f0e39 Mon Sep 17 00:00:00 2001
From: Daniel McKnight <daniel@mcknight.tech>
Date: Sat, 20 Jan 2024 04:18:51 +0000
Subject: [PATCH 2/2] Revise post

---
 2023-01-18_NAS_Setup.md | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/2023-01-18_NAS_Setup.md b/2023-01-18_NAS_Setup.md
index 34abe1d..8061b0a 100644
--- a/2023-01-18_NAS_Setup.md
+++ b/2023-01-18_NAS_Setup.md
@@ -148,6 +148,9 @@ match what I've described. At the moment, I'm replicating my new ZFS dataset to
 re-creating the pool on my NAS with the changes I discussed here (and with an additional drive in the data
 VDEV).
 
+Migrating my data to Unraid and back is essentially a test-run of my data recovery plan so I will plan on a
+dedicated post describing that process and any complications I run into.
+
 Once I have my data comfortably in its new home, I will flesh out my backup strategy and be ready to look into
 how I want to structure compute. I mentioned [in my first post](https://blog.mcknight.tech/2023/11/29/Homelab-Upgrades/)
 that I was considering Proxmox as a solution here, but I have since been hearing more about 
-- 
GitLab