procedure for mdadm in-place RAID5 to RAID6 upgrade

The answer "according to the book": Make and verify a backup, destroy your current raid, create the new raid exact as needed, restore, call it a day. Most likely this is the faster approach too, when considering the resync times.

The answer to your question: No, you do not have to manually adapt mdadm.conf when using mdadm.

"Since all SATA slots are currently in use, I cannot add the new disks while the old ones are still online."

I do not question the fact, but the conclusion. As mdadm is flexible and can manage arrays that include disks connected via different interfaces. This allows you to use temporary external storage (such as USB drives) to assist with the disk replacement process, even when all internal SATA slots are currently in use. So you have options.

Regarding your current plan: Overall Nahhhh. As said in the beginning, I would not do it this way in general. But my concern away for a minute and assuming the approach as described by you, then...

• Point 1 and 2 (first backup and then verify): Very good, always the correct start. Approved.

• Point 2,3,4: (unmount and shrink the filesystem): I think this steps are not necessary, as mdadm operates independent of the fs. The RAID rebuild process deals with blocks of data and parity, irrespective of how the fs is organized on top of those blocks. If you have a proper backup and your RAID array is in a stable state, you can proceed with replacing the disks one by one without shrinking or touching the fs. So you can skip this steps.

• Point 5: Never a fail, without a remove. Why? As this is important to ensure that mdadm understands that a disk is no longer part of the array, and it should no longer expect to use it for data storage or redundancy. Corrected:

mdadm --manage /dev/md0 --fail /dev/sdd1 mdadm --manage /dev/md0 --remove /dev/sd1 

• Point 7: No. Reducing the existing raid will fail. The --grow option can be used to increase the number of devices, but not shrinking. As far as I know, this is not supported by mdadm. (Please correct me someone if I am wrong on this point)

---

In case you really really really want to do an in-place replace, then:

Before you start, please google yourself the different rebuild times of Raid5 and Raid6, then do a quick calculation to decide if you really want to invest this much time. (You have been warned)

In case you still want to do it this way, and you accept the rebuild times:

• Make a backup
• Verify your backup
• Store the backup in a place which ensures that you will find it later, but so far away that the backup disk is not mixed up by accident with one of the other disks in the following procedure. (I warn out of my own experience here)

Then repeat the following procedure for disk sd[c|e|f|g], basically to replace every 4TB disk one by one with 16TB disks:

(Replace sdX1 with sdc1 in the first run, with sde1 in the second run, and so on...)

• Fail and remove the disk.

mdadm --manage /dev/md0 --fail /dev/sdX1 mdadm --manage /dev/md0 --remove /dev/sdX1 

• Power off, replace the old 4 TB disk sdX with the new 16 TB variant, power on.
• Create a partition on the new disk:

parted /dev/sdX mklabel gpt parted /dev/sdX mkpart primary 0% 100% 

• Add the new disk to the array:

mdadm --manage /dev/md0 --add /dev/sdX1 

• Monitor the rebuilding process

watch cat /proc/mdstat 

Wait until the rebuild is complete, before proceeding to the next disk. Continus with the next steps once all 4TB disks have been replaced with 16TB disks and the array is rebuilt.

• Convert the RAID 5 to RAID 6 and use the full capacity of the new disks. As you do not have spares to avoid a degraded array during conversion, you have to force this conversion step to happen.

mdadm --grow /dev/md0 --level=6 --force mdadm --grow /dev/md0 --size=max 

To be very clear and precise on this raid conversion point, in order to set the right expectations: This conversion from Raid5 to Raid6 will work technically just fine, and you will end up in a clean Raid6 state too. But still, the Raid6 will stay in degraded state, until you add a spare. I understood that you can not add a disk as spare, as all seats are full already in your system. So this is maybe (most likely) not what you want to reach.

• Resize the Filesystem. You said you are using ext4:

resize2fs /dev/md0 

---

HINT HINT HINT: Test out ideas in a safe space before proceeding with real data. mdadm is flexible and supports any type of block device, including virtual ones created from files.

This flexibility allows you to test any procedure in a controlled environment before implementing on real data. Here is how to replicate your current setup for your testing:

# Create a test directory mkdir -p /root/my_raid_tests # Create files to represent block devices, associate loop devices with these files for i in {1..6}; do dd if=/dev/zero of=/root/my_raid_tests/file$i.img bs=1M count=100 losetup /dev/loop$i /root/my_raid_tests/file$i.img done # Create the RAID 5 mdadm --create /dev/md/my_raid_tests --level=5 --raid-devices=6 --layout=left-symmetric --chunk=64K /dev/loop{1..6} # Verify the array mdadm --detail /dev/md/my_raid_tests # Create a filesystem on the array mkfs.ext4 /dev/md/my_raid_tests # Create a mount point and mount the array mkdir -p /mnt/my_raid_tests mount /dev/md/my_raid_tests /mnt/my_raid_tests 

And here is how to get rid of it:

umount /mnt/my_raid_tests mdadm --stop /dev/md/my_raid_tests mdadm --zero-superblock /dev/loop{1..6} for i in {1..6}; do losetup -d /dev/loop$i done rm -i /root/my_raid_tests/file{1..6}.img rmdir /root/my_raid_tests /mnt/my_raid_tests