I recently acquired two used blade servers and a short rack to put them in. I’m planning to use one or the other as the replacement for a media server that died on me a bit ago. The old media server was just a little refurb dell workstation, with a single SSD in it, but the servers have 6 and 8 bays, respectively.
I would like to RAID them so that one drive dying doesn’t lose any of my media, and I was leaning towards Ubuntu server as an OS. I’m not sure how to do that, and I’m kind of poking around for info and advice. Hit me with it.
It’s sounds like what you’re looking for is backup, and RAID is not backup, it’s redundancy to maintain uptime (as well as data integrity, and in some cases performance). I’d highly recommend you look into backup options, with best case being a 3-2-1 backup strategy.
To be fair, I’m being a little hypocritical. I’ve been working on my backup strategy for years and still don’t have any remote backups yet. Personally, I have a JBOD system, with 8 drives ranging from 2TB to 8TB, so my setup might be a bit complicated for your purposes. I’m not worried about uptime, and am focused on data integrity. I’m not using actual RAID because in the case of a catastrophic failure, I don’t want to lose all my data. I use snapRAID to create some redundancy, and I pool my data drives using mergerfs.
If you are still interested in RAID, I would recommend staying away from hardware RAID as I’ve commented in other places of this post. It has it’s place in data centres but really doesn’t make sense for consumers anymore. There is a lot of good advice in the rest of the comments about RAID, so I’ll summarize my thoughts. If you only plan on having 2 drives, RAID 1 is a good option, though it’s generally used for it’s write performance and that’s probably not necessary on a media server. My current server is running on decade old, lower-end, consumer hardware, and even in that extreme case, media sometimes takes a second to start 1080p content remotely. If you want to add drives and are willing to expand in redundant pairs, you you can either add another RAID layer (RAID 1+0), or pool the partitions together. If you want to be able to expand by single drives/have more than 50% of your potential storage be realized, you could look at RAID 5/6 or ZFS/btrfs. Note that for RAID 5/6, drives need to be equal size.
I’m mainly concerned about:
- Not losing data if one drive dies on me.
- Fast reads
- Easy plug and play expansion
Since I’ll have 8 drives (or 6, if I use the smaller server, it would be nice if I could swap out one of them without losing data and add a larger one, which would then get used automatically. Is that something that RAID is good for?
I’m hesitant to set up backups because it’s going to be a lot of data.
I’m mainly concerned about:
- Not losing data if one drive dies on me.
Sure, that’s what RAID is designed to do. However, I’d suggest also looking into what happens when your array is degraded and how to rebuild it.
- Fast reads
I’m a bit surprised you need fast reads with a media server. You’re probably going to have to clarify your needs a bit more.
- Easy plug and play expansion
Since I’ll have 8 drives (or 6, if I use the smaller server, it would be nice if I could swap out one of them without losing data and add a larger one, which would then get used automatically. Is that something that RAID is good for?
Standard RAID levels generally don’t have options to add larger drives. I’m not sure what you mean by “plug and play”. I’m pretty sure almost all setups will involve a fair bit of configuration.
I’m hesitant to set up backups because it’s going to be a lot of data.
It’s also a lot of data to lose if things go more wrong than you expected (multi-drive failure, bit-rot, etc.).
I did this yesterday for my media server on Debian and it was very easy. Use mdadm to create the raid. There are guides online on how that are very easy to follow. You may want to use partitions on the drives as another comment recommended (mdadm supports whole drives and partitions so do whatever you think is best). Next you should have a device /dev/mdX that you can format to your fs of choice. After that just use lsblk to get the uuid of the raid and mount it in fstab like any other drive.
you would want to use the hardware raid that likely already exists. its been a minute since i setup dell, but you should be able to boot into the raid controller bios (some ctrl-key sequence) and configure your raid there… then you just install whatever you want on the defined logical drives (linux/windows/hypervsior)
@originalucifer @blackstampede if you can just do software raid and if possible get the disks to look like JBOD (just a bunch of disks) CPUs are so much faster these days software raid even ZFS offers so much more than hardware raid.
i wouldnt on a non-jbod, retail server box. if this was a random workstation without onboard hardware raid, then sure.
im not sure how you think sharing the main processor with the raid when there is already a perfectly good set of processors for the raid is going to be faster.
@originalucifer @blackstampede I’d rather ZFS for the data integrity stuff than anything else.
what specific feature of ZFS are you frothing over to sacrifice your primary processing for it?
the hardware raid in this box was designed for business and would be more than adequate for the requested purpose
You’re right, hardware RAID still has some use for businesses, but it’s generally a bad idea for consumers. The main reason is the procedure if the RAID controller fails. In commercial applications they have spare, compatible controllers, so a quick hardware swap and you’re back up and running, you don’t even need to rebuild the array. However, consumers generally don’t have a spare controller, and if they don’t, they can’t just get any controller, they need a compatible one or the array is lost. If a system running a software RAID has a hardware failure, the array can be moved to a new host and mdadm can rebuild the array without needing specific hardware.
but this guy is specifically not using consumer hardware
Yes, but they’re using it in a consumer setting. That was the whole point of my comment. It sounds like they may have 2 identical RAID controllers, which means they might have a spare. However, if one dies, they’d be looking at obtaining another spare, migrating their data to a new setup, or risking complete data loss.
Hardware raid is missing many features of modern software raid like ZFS. Expansion is harder, replication and snapshotting options don’t exist like they do on a COW raid, speed improvements with ZIL and caching aren’t there, the list goes on.
Ubuntu supports ZFS very well. And if you’re going software raid, for the love of dog, don’t use md. It’s ancient.
HW raid also screws you if the controller dies.
There’s nothing wrong with MD. It’s old but also rock solid. It’s super flexible, can do array configurations that can’t be done in other ways. And most importantly it decouples redundancy from other features and allows you to pick and choose your feature set.
Is there a way to checksum in md?
Depends what you mean by it. Some of the traditional RAID levels use stripe parity. If you mean file checksums then no, mdadmin only deals with disk devices, doesn’t deal with filesystems and below. You can use filesystems with built-in checksums or other methods. For example I use RAID1 and I take incremental backups with Borg.
