Anatomy of a homelab RAID10

Most of my homelab's interesting storage lives behind a single mount point: /mnt/data. It's an 8 TB ext4 filesystem on a four-disk mdadm RAID10, exported into Kubernetes as a hostPath volume that Jellyfin, the *arr stack, a couple of game servers, and a Matrix homeserver all share. Nothing exotic — but I get asked about it enough that I figured I'd document the build as it stands today, warts included. This pairs with the homelab storage and backups overview; this post is the close-up on the one array.

The hardware

The host is a personal box that's been around long enough to accumulate three different generations of WD Red 4 TB drives. Storage controllers are split across two paths:

• AMD 400-series chipset SATA controller (motherboard onboard)
• Broadcom/LSI SAS2008 HBA ("Falcon"), the classic homelab card from the IT-mode-flashing era

All four data drives hang off the LSI HBA (they show up with sas transport in lsblk), which sidesteps the chipset's port limits and motherboard quirks. The onboard SATA ports carry the boot SSD and a scratch disk that isn't part of the array.

NAME  SIZE MODEL                SERIAL          TRAN ROTA
sda   3.6T WDC WD40EFRX-68WT0N0 WD-WCC4E1••••••  sas    1
sdb   3.6T WDC WD40EFRX-68N32N0 WD-WCC7K6••••••  sas    1
sdc   3.6T WDC WD40EFRX-68N32N0 WD-WCC7K3••••••  sas    1
sdd   3.6T WDC WD40EFRX-68N32N0 WD-WCC7K1••••••  sas    1

Power-on hours tell you this array wasn't built in one shot:

sda is a generation younger (the -68WT0N0 revision) and clearly a previous replacement. sdb and sdc are siblings bought together — same WCC7K serial batch, hour counters 32 apart, so they spun up within a day of each other. sdd is the oldest, bought separately and second-hand at the time. You can build a matched-batch array and worry about correlated failure, or a salvaged-mix array and worry about everything failing independently at different times. I got the second kind, mostly because that's what the parts bin gave me.

The array

md0 : active raid10 sdb1[3] sdd1[0] sda1[1] sdc1[2]
      7813769216 blocks super 1.2 512K chunks 2 near-copies [4/4] [UUUU]
      bitmap: 0/59 pages [0KB], 65536KB chunk

The choices that matter:

• RAID10, near=2. Two mirrored pairs, striped across them. Survives any one drive failure; survives two only if the two dead drives aren't the same mirror pair. Better random-write performance than RAID5/6 and no parity recompute on write — which matters when half the workload is media-metadata churn and the other half is game-server saves.
• 512 KB chunks. The mdadm default for raid10, fine for large-file media. I haven't tuned it.
• Internal write-intent bitmap. Lives on the array itself; after an unclean shutdown it only resyncs the dirty regions instead of all 8 TB. Small write-perf cost, trivially worth it on a media tier.
• Metadata 1.2. Stored near the start of each member partition, so the array assembles without scanning the whole disk.

One line in mdadm.conf lets it auto-assemble on boot:

ARRAY /dev/md/0  metadata=1.2  UUID=294eeaad:0a5aa2bb:b373b123:e7877cff  name=oldhost:0

(The name=oldhost:0 is a relic from the previous host this array lived on — mdadm doesn't care, and I've never bothered to rename it. I've masked the literal old hostname here.)

A pairing detail worth knowing

In a four-disk near=2 layout the mirror pairs are the adjacent RaidDevices: (0,1) and (2,3). Map that onto the members and you get pair A = sdd + sda and pair B = sdc + sdb. That's worth staring at:

• Pair A is the 6.8-year drive mirrored with the 2.7-year one — nicely mixed, low correlated risk.
• Pair B is the two matched-batch siblings, sdb and sdc — bought together, ~32 hours apart, the exact correlated-failure setup you don't want both halves of a mirror to be.

flowchart TD
    MD["md0 — RAID10 near=2, striped across two mirror pairs"] --> A["Mirror pair A"]
    MD --> B["Mirror pair B"]
    A --> A1["sdd — 6.8 yr"]
    A --> A2["sda — 2.7 yr"]
    B --> B1["sdc — 5.7 yr, same batch"]
    B --> B2["sdb — 5.7 yr, same batch"]

    %% color = correlated-failure risk: green pair is age-mixed, amber pair is a matched batch
    classDef mixed stroke:#a3be8c,stroke-width:2.5px
    classDef risk stroke:#ebcb8b,stroke-width:2.5px
    classDef plain stroke:#7b88a1,stroke-width:2.5px
    class A1,A2 mixed
    class B1,B2 risk
    class MD,A,B plain

I didn't plan that; the array assembled in whatever order the disks enumerated. If I were placing them deliberately I'd split the batch so each mirror has one sibling and one outsider. Cheap insurance against two near-identical drives dying the same week. (For surfacing that kind of latent risk before it bites, see monitoring your homelab.)

The filesystem and mount

ext4, created when I last rebuilt the host. The features list has one option worth calling out:

Block size:    4096
Checksum type: crc32c
Features:      has_journal ext_attr resize_inode dir_index filetype extent
               64bit flex_bg sparse_super large_file huge_file dir_nlink
               extra_isize metadata_csum

metadata_csum — checksumming on filesystem metadata, on by default in recent e2fsprogs. It won't save you from bit-rot in file data (that's a ZFS or btrfs job), but corrupted directory structures and inode tables get caught early instead of silently. Worth the small CPU cost.

The fstab line:

UUID=50110a6b-...  /mnt/data  ext4  defaults,nofail  0 0

Two opinionated bits:

• nofail — boot continues even if the array doesn't assemble. On a box with a degraded array the alternative is dropping to emergency.target with no system at all, which I've lived through once and didn't enjoy.
• 0 0 (skip boot fsck) — boot-time fsck on an mdraid array that's still assembling or resyncing is fragile. I run fsck by hand when the situation actually calls for it.

Mounted relatime (the default), the right call on a media tier: you don't want an atime write for every Jellyfin scan, but you do want it updated occasionally for tools that check it.

/dev/md0  ext4  7.3T  6.4T  522G  93%  /mnt/data

93% full. Yes, I know.

What lives on it

/mnt/data is a hostPath mount for Kubernetes pods, organised into per-application subdirectories, each owned by the application's UID:

• jellyfin — media library (the bulk of the space)
• sonarr, radarr, prowlarr — *arr configs and metadata
• matrix — homeserver media store
• game-server data and workshop addons
• smaller things: home automation, a Forgejo instance, odds and ends

Nothing on this array is irreplaceable, and that's the implicit budget that makes a four-disk RAID10 of second-hand WD Reds an acceptable risk tier. Everything I genuinely cannot lose lives elsewhere — small, off-host, encrypted, backed up properly. Drawing that line carefully matters more than chasing nines on the array itself.

What I'd do differently

1. Split matched-batch drives across mirror pairs. As above — pair B is two siblings. Each mirror should have mixed-age drives.
2. Wire up mdadm --monitor to an address I actually read. The default MAILADDR root on a headless box with no MTA is a silent monitor — it looks configured and tells you nothing.
3. Run smartd with weekly short tests and monthly long tests, piped to the same notification channel.
4. Schedule periodic mdadm --action=check on cron, to surface latent bad sectors before a real read trips over them.
5. Leave more headroom. A 93%-full ext4 is fine; a 93%-full ext4 that's also rebuilding while an import writes a 60 GB rip is asking for a bad evening.

That's the whole tier: four spinning disks behind a SAS HBA, an mdraid that just works, an ext4 that's pragmatic and unfashionable. It sits under the rest of the homelab on Debian build. The next post is about what happens when one of those four drives — the 6.8-year-old one — starts having opinions.