![](/static/253f0d9b/assets/icons/icon-96x96.png)
What has prompted your interest in data hoarding?
Censorship and Memory-holing
What has prompted your interest in data hoarding?
Censorship and Memory-holing
I can’t tell you how many channels have disappeared and been memory-holed. Especially since censorship went into overdrive around 2019.
Data hoarders can show you how the world was before all that happened.
Unraid and Proxmox
Yes and no.
Yes if you have the resources to monitor and update. Companies have entire teams dedicated to this.
No if you don’t have the resources/time to keep up with it regularly.
IMO, no need to take this risk when you have services like Tailscale available today.
It’s not cheap to operate a business in Canada
This is very complicated to just give an answer, because:
It varies greatly based on the content. Animated compresses vastly differently than an action movie.
Varies greatly based on encoder. NEVC vs CPU etc
Varies greatly based on encoder options. I.e. -b:v -minrate -maxrate vs -rc vbr -qmin -qmaxcq values, etc
Varies greatly based on who is watching, the TV they use and their tolerance and experience.
Savings are greater at 4k than 1080p. But once you start adding HDR into the mix, you’re in a whole new world.
Even the people with very discerning eyes can’t agree on everything related to this topic. Wish I could just tell you do x… but you’ll have to test various methods and determine what you are happy with.
or, if you just want some space savings… use some default setting that cuts it in 1/2 and forget about it.
Really depends on the gear. Some of it you can, some you can’t.
Check IPMI and see if you can adjust it there. For mine, I use the IPMI plugin that you can get for Unraid. So some OS’s do have solutions. But depends on your OS and hardware.
You can replace fans, usually. But need to make sure you still have enough airflow for the equipment.
I did replace my server grade fans with the Noctua Industrial. They are still a bit loud for being in the same room, but quiet one room over. Where before you could hear it in any room of the house.
My goal was quiet one room over.
Sure, I have hit 10TB/mo. I’m making sure I get my money’s worth for paying $10 to have unlimited with comcast.
I think in general they have lower RPM and run a bit cooler and use a little less power. That usually comes with a bit less performance.
But I’m hooked on the WD Ultrastar series. Server Grade and fast. Also has low power usage, at full tilt, mine use less than 10w each. I’m running 20 hc530’s and been rock solid.
Really depends on your use case.
Unraid is wonderful and easy to use. But really has two reasons to use it:Unraid Array fits your file storage strategy. (few writes, mainly reads)You want an EZ way to get into docker and use the unraid appstore.
Other than that, you can probably find everything on Debian or Ubuntu. (I prefer Debian for services)
You can add one more:
Proxmox and then run a Debian VM for docker, for example, and compartmentalize other things you may want to run.
Also, download a mem test utility and run it overnight to test your hardware.
I started out self hosting everything… 20 years ago or so. Then I got swept up in the “cloud” movement and put so many things into the “cloud”.
Today I’m reverting back to how I started, self hosting everything that I can.
Mainly privacy, but also because they keep changing and I don’t want to have to worry about them.
I just didn’t feel like it was ‘my’ data anymore.
X12SDV-4C-SP6F
Man, I have been looking into the X11SDV-8C-TP8F and I’m so close to pulling the trigger. I love the form factor.Do you know of any with 25g networking that also has EDIT: “QuickAssist” enabled like the X11SDV-8C-TP8F ? The jump from 4 core to 8 core on those adds QAT
I looked at those cases, ended up going with this because easier to expand and hte PSU size:
Can find them for less elsewhere:
https://www.amazon.com/gp/product/B095YMXW1K/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1
Have a look at these HDD’s, I have seen them as low as $150. i run 20 of them right now, some have 3 years run time now. Been very happy with this vendor, usually only a few hours of runtime:
https://www.disctech.com/Western-Digital-UltraStar-DC-HC530-WUH721414ALE604-0F31156-0F31284-14TB-3.5-7.2K-RPM-512e-SATA-6Gb
Storage type:
If you need the speed, stick with ZFS.
But it should depend on the data and how risk averse you are (how easy to replace the data is). I started all ZFS, now I use this method:
Unraid Array. For easily replaceable media, like movies. (Where I have a list and can easily re-download/upload) I use only the Unraid Array for that. Mainly for data that is write once read often.
Downside:
Unraid write speed is slow. No scrubbing.
Upside:
only the 1 disk that has the data spins up. This saves me about 180-200W of power and lots of wear on the drives.
It’s also very storage efficient, I only run 2 parity disks with 20 drives. I wouldn’t do that on a regular raidz. But with unraid data lives on each disk, so you don’t loose the entire array if you lose more than 2 disks. This changes the risk math.
Then for all my critical data, that I need ZFS speed and scrubbing for, I setup a raidz1 pool with 4x Intel P4510 4TB. (Home pics/media)
Those I got for $200/pc new on eBay.
Can be safer. Can be worse.
A poorly configured self hosted vaultwarden can be a major security issue.
A properly configured one is arguable safer than hosting with a 3rd party. Lastpass taught me that one.
If you configure it to where it’s not exposed to the web, and only accessed through a VPN, like Tailscale. It can be quite robust.
self hosted git repository.
I setup gitea on my server and use it to track version changes of all my scripts.
And I use a combination of the wiki and .md (readme) files for howto’s and any inventory I’m keeping, like IP addresses, CPU assignments etc.
But mainly it’s all in .md formatted with markdown.
I do this at the file system level, not the file level, using zfs.
Unless the container has a database, I use zfs snapshots. If it has a database, my script dumps the database first and then does a ZFS snapshot. Then that snapshot is sent via sanoid to a zfs disk that is in a different backup pool.
This is a block level backup, so it only backs up the actual data blocks that changed.
I don’t use photoprism, but have experienced similar in other docker containers. What is most likely happening is that something, like headers/ports, needs to be forwarded by NPM, usually b adding additional config to the “advanced” tab in NPM.
Sorry, I’m not familiar enough with photoprism to know what exactly needs to be added to the config, but I since nobody has replied, I thought it might at least give you a direction to search in.
Storage