• 0 Posts
  • 6 Comments
Joined 9 months ago
cake
Cake day: October 25th, 2023

help-circle

  • Even if you ran a basic sqlite nexcloud, if properly optimized, you can deal with millions of files like its nothing. And that is the issue, the bugs and lacking optimization…

    4650g + 64GB ram + Mysql and it was file locking on just a 21k 10GB folder constantly.

    I have written apps (in Go) that do similar and process data 100 times faster then nextcloud. Hell, my scrapers are faster then nextcloud in a local netwerk, and that is dealing with external data, over the internet.

    Its BADLY designed software that puts the blame on the consumer to get bigger and better hardware, for what is essentially, early 2000 functionality.


  • Any media you want to long term store, you want to rotate. Aka, move data from A, to B, to C, B to C to A, C to … Any storage media not specific designed with redundancy in mind, will see bit rot over time, depending on humidity, heat, location (if you live 2k high vs sea level), etc.

    I always moved my media around like that, never had a issue with data that goes back 20 years.

    For me any media is no different then having a car parked out for 6 months. Sure, it will work, but your battery may (will) be death, you brakes may get locked up, etc… Or leaving a house or apartment unattended for 6 month.

    SSD for cold storage is just as good. Been using some SSDs going back to when 256GB was expensive. But like with HDDs, Tapes, etc… rotate and rewrite.

    One of the biggest mistakes i have seen in a government job, was making backups, never checking / rewritting those backups and the day they needed those backups. Well, the taps had damage (despite only being 2 years cold stored). So the loads of money they spend on a dedicated guy, who’s job it was to do backups every day, the expensive tap machines, etc, was all for nothing. Simply they did not do checks/rewrites. And nobody was able to blame him, because he was following the exact instructions for making those backups. He even brought up the issue but … “instructions are made by people more qualified then you”. O, they tried to blame him but he stepped to the union and it was quickly resolved in a “nobody was to blame” (because can not blame the managers above him, now can we ;0 ). His instruction got changed to: Verify every tape at minimum X times per year, and rewrite on new taps, with backups for every tape (also increased budget to buy more tapes instead of recycling). And that was like i said, government ;)

    Any storage media will fail, and its up to the end user to ensure you have regular rewrites, checks, recoverable parity of the files itself, … Say this as somebody that lost months of programming work, when his primary HDD failed and multiple backups ended up useless ( in the old floppy days ). Multiple backups helps but even that can be fraud. I have seen written DVDs go bad all at the same time (one reason i never backup to DVD or Bluray)…

    So far i found SSDs more reliable then most people realize. As long as i rewrite them, every so often.