I have a load-bearing raspberry pi on my network - it runs a DNS server, zigbee2mqtt, unifi controller, and a restic rest server. This raspberry pi, as is tradition, boots from a microSD card. As we all know, microSD cards suck a little bit and die pretty often; I’ve personally had this happen not all that long ago.

I’d like to keep a reasonably up-to-date hot spare ready, so when it does give up the ghost I can just swap them out and move on with my life. I can think of a few ways to accomplish this, but I’m not really sure what’s the best:

  • The simplest is probably cron + dd, but I’m worried about filesystem corruption from imaging a running system and could this also wear out the spare card?
  • recreate partition structure, create an fstab with new UUIDs, rsync everything else. Backups are incremental and we won’t get filesystem corruption, but we still aren’t taking a point-in-time backup which means data files could be inconsistent with each other. (honestly unlikely with the services I’m running.)
  • Migrate to BTRFS or ZFS, send/receive snapshots. This would be annoying to set up because I’d need to switch the rpi’s filesystem, but once done I think this might be the best option? We get incremental updates, point-in-time backups, and even rollback on the original card if I want it.

I’m thinking out loud a little bit here, but do y’all have any thoughts? I think I’m leaning towards ZFS or BTRFS.

  • ShadowAEnglish
    3·
    5 months ago

    I couldn’t count the number of failed sd cards I’ve seen across all my fingers and toes.

    I’ve seen like 4 ssds in my entire life fail. Plus you could just do mdraid 1 / btrfs across 2 of them if you want

    • Avid AmoebaEnglish
      11·
      5 months ago

      Any failures of SanDisk Extreme Pro / Samsung Evo Plus?

      I buy the redundancy argument. I’d still use ZFS for that if possible though. 😂 All my machines use mdraid 1 for their system drives but now that I know enough about ZFS, I’d likely use it on root next time around.

      • ShadowAEnglish
        3·
        5 months ago

        Yes. I’ve always splurged on nice cards for my personal stuff. I think it’s more about the write behavior of Linux than anything else, since I’ve never had a card die in my camera.

        I refuse to use a pi with SD at this point. Saving $50 isn’t worth my time to reinstall things.

        • Avid AmoebaEnglish
          21·
          5 months ago

          You’re making me nervous.

          But then again, all my Pis are OpenWrt now which barely writes, so I’m probably fine. 😅