An Anxious Engineer's Digital Diary

Building a Server

Konstantin Andrikopoulos January 14, 2025 #homelab #server

Happy new year everyone! I actually manage to keep myself consistent and publish a blog post also in 2025!

Starting a Homelab

So recently I realized that my method of backing up to an external hard drive, was starting to be quite cumbersome. Not only was it annoying to have to plug the hard drive every time, it was also not cool!

So I decided to build a NAS so that I could easily make back ups over the network. This would be the first time I would be spending serious time setting up a proper homelab.

And of course I started overthinking it.

My initial plans were to build a low-grade server that would be my NAS. Thus it could be cheap and not have much computing power. Just enough to run TrueNAS, so I can save my files.

But what I also wanted to do was to build a separate server, with more computing power, which I would use to run actual services on. I wanted this second server to have no storage on it. Instead it would get it from the NAS, using iSCSI.

You will soon see that this plan did not come to fruition.

Choosing components for the NAS

Since this was the first time I was building a server since a long time, I decided to start with the cheaper NAS first. So any mistakes I would do would probably be less impactful.

Picking the Motherboard

My strategy was to pick a motherboard according to some constraints I had, and then buy components that would be compatible with it. This way, I imagined, would reduce my search space.

My constraints for the motherboard were:

As for the socket, I wanted to use an AMD CPU, and also I wanted to be able to find an older CPU to keep the costs low. Thus I opted for AMD's socket from the previous generation, and not AM5.

Since I saw that all these constraints really reduced the choice of motherboards (specifically the ECC RAM and IPMI), I realized that the motherboard would probably be one of the most expensive components of my build.

In the end I picked AsRock X470D4U2-2T, which satisfied all my constraints.

Along with it I bought the first 2 16GB DDR4 ECC RAM at 3200MHz I found online. I really didn't spend any time searching for the best RAM I could find.

I also bought a 550W PSU with Platinum 80 PLUS rating. I also didn't spend much time looking exactly how many watts I needed etc. I just wanted an efficient PSU.

Choosing a CPU

Here I wanted to keep the cost as low as possible so I initially picked a Ryzen 3 3100 for like 70 bucks.

However, for some reason the CPU would take a very long time to ship. Around 2 weeks longer than the other parts.

After waiting a while I saw online a Ryzen 5 5500 almost the same price as the Ryzen 3 I picked, with next day delivery! So I cancelled the Ryzen 3 and bought the Ryzen 5.

Hint: This would be a mistake.

Onboard 10GbE Was a Bad Idea

If you had checked the linked motherboard above, you would have seen that the onboard 10GbE comes as 2 RJ45 ports. And if you are better at networking than I am you must have realized that this was not a great choice.

You see 10GbE with RJ45 is... not great. Not only do the connectors get extremely hot, they also consume a lot of power (for a network connector).

Thankfully there was an easy solution. There is another version of the motherboard I bought, the AsRock X470D4U.

The difference with the other one, is that instead of the 2x10GbE RJ45, it has only 1GbE ports, but one more PCIe3 x4 slot. I can use that slot to plug a 10GbE NIC with an SFP+ port, so I can use fiber instead of copper.

I chose the Asus XG-c100F.

Buying Storage

One of the most important components for a NAS is for of course the storage. Since I was planning to have 10GbE, I thought that it would be ok to not have the fastest disks available. I could saturate the network even with a few SATA disks.

However, it was black Friday and I found for sale 4 Samsung PRO 990 2TB SSDs. Since the motherboard doesn't have NVME connectors, I also bought ASUS Hyper M.2 X16. This card has 4 NVME connectors, and plugs into a PCIe3 x16 slot. It requires however that you bifurcate the PCIe slot as x4x4x4x4.

PCIe bifurcation splits a single PCIe slot into multiple smaller ones, allowing connection of multiple devices to a single slot. For example, an x16 slot can be divided into two x8 slots or four x4 slots.

It is important to note here that not all CPUs support that. Specifically AMD CPUs with an integrated graphics card do NOT support x4x4x4x4 bifurcation. They can only do up to x8x4x4 instead.

AMD marks CPUs with integrated graphics by appending a "G" in the CPU name. Since I had bought a Ryzen 5 5500, this meant that there is not integrated graphics, and full bifurcation is supported. Or so I thought.

Putting Everything Together

After some more time all the parts arrived and I was able to put everything together. It was surprisingly easy and the computer started first try!

Only that it was not POSTing. Turns out, that it was not my fault, but the motherboard came with an older firmware version which did not support the CPU I had bought. After a quick update (using the IPMI UI) the machine POSTed, and I was able to get into BIOS.

There everything was recognized as expected. Except from the SSDs.

There I saw only one SSD. When I explored the menu a bit more I saw an option to manually select how you want to bifurcation the PCIe slot, but the best option I could see was x8x4x4 bifurcation. This made 2 SSDs visible in the bios.

Trying to Bifurcate

This is where I got lost and spent the most time trying to figure out the issue. I unplugged everything, and just left the bare minimum to see if something was messing with the bifurcation options. I searched every BIOS option. But I had no luck. The option for x4x4x4x4 bifurcation was nowhere to be found.

After a couple of days I saw that the CPU I bought comes from the Cezanne die. And that die was used to create CPUs with integrated graphics.

So I found this AND the fact that I could only see bifurcation up to x8x4x4 highly suspicious. I then searched a bit more online and managed to find a blog post (for which I don't have the link any more) that also confirmed my suspicions that my CPU does not support full bifurcation.

My current theory is that the Ryzen 5 5500 is a result of binning. Probably it is a batch of faulty CPUs with integrated graphics. The APU was not working properly, so they decided to sell it as a CPU without integrated graphics.

But anyway, I bought a Ryzen 7 5800X, because it was used by a colleague and they could verify that it supports full bifurcation. Also it was available for next day delivery, and at that point I didn't want to wait anymore. I wanted to have my toy!

Conclusion

Well for me that was fun. Building a server was something that scared me since I didn't have any experience with it. But I managed to do it, and now I want to also make a gaming PC. I wonder what can go wrong there!

But it is not just fun. It is a small engineering problem. You need to make sure all the parts fit together, and they will be delivered in time. You also see how you can trade off money for delivery time, or for time spent trying to find the best and cheapest option.

The fact that I had to correct my RJ45 mistake after I had bought some of the parts was also interesting. The solution was simple in my case, but it shows how engineering is all about solving a problem given specific constraints, which can change at any time during the project.

Similarly with getting a different CPU. Now that the final server that I have built differs quite a bit from what I first planned, I also had to re-asses my initial idea of having a separate NAS and server. Since I have quite a bit computing power, I can probably combine the two in a single machine. At least for now. If I see that I am actually utilizing my server, I might build a second one.

But also it renewed my awe for computer engineering. I ordered different parts, from different shops and different manufacturers. And when I put everything together they just... worked? Isn't this amazing?

I mean it sounds super trivial to someone who has more experience than me, but think of all the engineering effort it took to make this possible. From the inception of computers to today.

Anyway, thanks for letting me share this story with you!