Self-hosting a few services on one’s own hardware is a great way to wrest some control over your online presence while learning a lot about computers, software, and networking. A common entry point is using an old computer or Raspberry Pi to get something like a small NAS, DNS-level adblocker, or home automation service online, but the hobby can quickly snowball to server-grade hardware in huge racks. [Dennis] is well beyond this point, with a rack-mounted NAS already up and running. This build expands his existing NAS to one which can host a petabyte of storage out of consumer-grade components.
The main reason for building this without relying too much on server-grade gear is that servers are generally designed to run in their own purpose-built rooms away from humans, and as a result don’t generally take much consideration for how loud that environment becomes. [Dennis] is building a lot of the components from scratch for this build including the case, the backplanes for the drives, and a backplane tester. With backplanes installed it’s time to hook up all of the data connections thanks to a few SAS expanders which provide all of the SATA connections for the 45 drives.
There are two power supplies here as well, although unlike a server solution these aren’t redundant and each only serves half the drives. This does keep it running quieter, along with a series of Noctua fans that cool the rest of the rack. The build finishes off with an LED strip which provides a quick visual status check for each of the drives in the bay. With that it’s ready for drives and to be connected to the network. It’s a ton of wiring and soldering, and great if you don’t want to use noisy server hardware. And, if you don’t need this much space or power, we’ve seen some NAS builds that are a bit on the smaller side as well.
Title feels a bit misleading when he mentions the drives are enterprise grade.
You dont have to be an enterprise to use enterprise drives. I’m using several here at home.
enterprise drives are not consumer grade
Consumer drives are useless for this application. They shake themselves to death if you put more than 8 in a single chassis.
Pack that many consumer grade drives into one box and they’ll shake each other to death. The remainder of the build is largely off the shelf, aside from the SAS expanders which are fairly inexpensive at this point on the used market, being several years out of date. They to are irreplaceable, being as sata expanders essentially don’t work and SAS equipment of this vintage is cheap and plentiful.
non redundant power supplies powering half of the array what could go wrong.
Also, the article states that having two power supplies supplying half the drives each makes the system quieter. I’m wondering how it makes it quieter.
Dynamic fan speed. More load=more heat=faster, louder fans. Split the load and the fan speed requirement drops.
Less load on each power supply means less fan activity.
Yes, but 2 sources of noise instead of one. I’m unsure if sound level linearly adds up, but i would say that 2 producing half the noise = the noise of a single fan.
Noise is log based, two sources of the same db means an increase of 3db.
Nah. There’s a rule of thumb that doubling the noise sources increases the volume by about 3 dB (decibels), where 10 dB (nobody calls it a bel…!?) basically sounds twice as loud; It’s logarithmic. It works a lot like the perception of brightness, insofar as someone turning on a second light doesn’t make the red seem twice as bright.
*room
Apologies for the typo.
Fans are also famously nonlinear, with lots of fun cubic terms. Running them slower can make them SIGNIFICANTLY quieter.
Use the right durability encoding while distributing the data across the drives correctly and you can lose half your drives and continue serving data.
A simple ZFS mirror would suffice, but in this case it’s a set of RAIDZs. At any rate, this isn’t a 5-9’s facility. Some downtime to replace a failed PSU is tolderable.
I have not watched the video (yet). But my initial thought is that large enough spinning rust storage will be helium filled. Helium filled disks have an expected lifetime of 5+ years before the helium leaks away causing in effect a vacuum to weld the heads to the platters. So I’m having a hard time time thinking of a home lab use case of 1 PiB of storage with a shelf life of a little over 5 years (regardless if it is powered on or off).
Not commenting on the project, but as regards the use case…
My employee pointed out recently that he could get a 30 TB drive for about half a grand. That’s 30,000 gigabytes, which is roughly 30,000 hours of video.
Assuming you sleep 8 hours a day, that’s 5 years of continuous waking video. Assume you use a filter or compression system that clips or speeds up boring parts of your day (such as driving or eating or working out at the gym). After the video is recorded, it reviews the the video and speeds up or downgrades the format if there’s nothing interesting going on.
With that much storage capacity it’s entirely possible to have a video recording of your entire life. With smart glasses, you could record all of your interactions all of the time.
I can see some use cases for this. I’d love to have a “history search” function that shows me which websites I’ve viewed, which scientific papers I’ve read (I’ve since taken to storing a copy of everything I read, for later reference), which people I’ve met (and where), what someone actually said in a conversation, and so on.
The idea of having a “black box” system on your belt the size of a paperback that records the entirety of your life would lead to some interesting social changes.
Additionally, I was writing a short story about the future and asked ChatGPT how small of a volume the library of congress could be stored so that an astronaut could carry it around with him.
I was surprised to learn that we could do that today. The text LOC is only about 10-20 TB (I’ve got discs that can hold that already), and if you want to include movies and images it’s only about 20 PB.
Easily stored on your rocketship to take to a distant planet.
About 50 years from now everyone could be carrying around what is essentially the entirety of human knowledge, plus an AI that can rapidly access the data and summarize it for you.
Like I said, interesting social changes.
Sounds like https://en.wikipedia.org/wiki/The_Entire_History_of_You
That is absolutely not the failure mode of helium drives, the exact opposite happens where air ingresses into the drive and causes the heads to move the opposite direction.
Some drives even have a SMART value to display how much air has entered the drive by indirectly measuring how viscous the atmosphere inside the drive is via spindle torque/current.
The primary cause of the air ingress is heat cycles, not shelf time or run time. It is reasonable to expect many decades before a failure from helium contamination causes a drive to fail; platter delam or other mechanical issues are more likely to happen in this time so unless your constantly spinning up your drive and then putting it to sleep helium isn’t an issue.
If you’re constantly cycling the drive you’ll wear out the plastic wedge the heads park on far more quickly. Those are typically rated in the hundreds or thousands of cycles.
The solution is obviously to put the whole array in a helium-filled cabinet, where you can easily compensate for the helium leaking away. :)
And another helium cabinet around that one too, just in case.
This could only work if the oscillator on the drive’s PCBA isn’t of the MEMS variety:
https://hackaday.com/2018/10/31/helium-can-stop-your-iphone-maybe-other-mems-too/
Something adjacent to this sort of came up in conversation for me recently when discussing the feasibility of using HDDs in space in a non-crew compartment environment; Basically what it came down to was that helium-filled drives would be *more viable than the air filled drives because they are vaguely hermetically sealed so they should last at least a few months in space because head crashes ensue from the helium leaking out.
Yeah, no! If the derives are enterprise grade then the title is a lie. All disk fail, but do this with consumer grade and let us know how long befor eyou lose the data.
Consumer drives fail quickly!
15 years ago my old company built a half petabyte storage array – across multiple (20?) computers as drives were smaller and so we needed a lot of drives! Plus we wanted the processing power.
We quickly found that at any point several drives would fail ECC checks because single bit errors were common. Most users would never know because they don’t hash and check their files, and a 1-but error in a photo or video (which is what most large data is/was) isn’t noticeable.
Then shortly after we found that the MTBF of consumer drives plummeted when they’re run 24/7 in a box with 7 other drives. Heat wasn’t the issue, it was vibrations.
It was still a useful project and mostly served our purposes, but learnt a lot of lessons!
Top tip: always buy enterprise drives.
You cannot use consumer grade drives for this. Full stop. They do not tolerate the vibration of that many drives in one cabinet and will shake each other to death.
If you have a problem with whether this is made using “consumer grade” parts, I suggest you compare the total bill of materials cost of this disk shelf against similar commercial options.
Without watching the video, I’d guess the design is based on one of Backblaze’s storage pod designs.
I know that at least the earlier ones had their design files published by them, so anyone could replicate their (then?) in-service design(s).
If a drive fails on average once every 15 years, with 45 of them there’s going to be a failure every 4 months.
Failures follow the bathtub curve. They’re not randomly distributed in drives of similar age.
And here I sit with 1.3TB used on my server wondering what I’ll ever use the rest of 8TB for….. Yikes! :eek:
download AI models to have them just in case but also do some P2P
My 3TB hard disk filled up completely last night after recording the first half of a movie.
The half with the FBI warnings, and the ads for further movies.
I’m recording unencrypted DVB-C.
Correction… 2.3TB used. Getting old … And half of that is multi-media files… Only 1.1TB is actual home data that I’d like to keep for the time being.
Anyone who has to store a lot of video has this problem, most commonly people doing video editing and production. There are actually lots of videos of youtubers building large storage servers with 10Gbps ethernet so that scrubbing is smooth from their editing workstation.
Not much to grab on when you want to pull a drive out.