Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- I can't tell you what works for "you", but here's my setup, with the hope it inspires someone:
- Live data lives in the cloud. I could self host it, but it would always be inferior to the cloud offerings, and usually more expensive for my ~3TB data.
- I make local backups nightly
- I make remote backups nightly to another cloud.
- Once every week I make a local backup to a device that is offline 6/7 days a week (an old Synology NAS that powers on automatically, and powers off when it's been idle for 30 minutes).
- Every year I curate our photo library and burn a set of M-Disc Blu-Ray copies of all photos created or modified in the past 12 months. Two identical sets, one stored at home, the other stored at a remote location, each clearly labeled "Photo Backup <year>".
- Every year I also update a couple of external HDDs with the entire photo library, again identical copies, contents are verified yearly and updated, and stored again. Disks are also clearly labeled as "Photo backup".
As others have written, curate your data. In my case we have a 2.5TB photo library spanning a couple of decades, but that could easily have been 4TB without curation. I only backup documents in the nightly/weekly backups. (Personal) Documents usually only hold value for a short time, and after that it's mostly sentimental.
Anything media, or anything downloaded from the internet like books, music, etc, regardless of if I purchased it or pirated it, is not getting backed up. If it came from the internet, there's a good chance it can still be found on the internet.
I also (mostly) don't run RAID. RAID is for availability, and since all my important data lives in the cloud, and I will likely survive if one of my backups dies, there's little reason to run raid. The only exception is the share where PhotoSync backs up our photos, which is on a "small" RAID1 volume mainly because it acts as the source of all the other photo backups, so consistency and correctness is important.
by 8fingerlouie - The best backup solution that I found is ZFS send (even if it’s not strictly a back up tool).
ZFS send raw encrypted stream from laptop to a server automatically. Encrypted compressed incremental. No need to verify: if snapshot exists at destination, there is no error. If a problem is encountered, restore from redundancy. Errors can be detected and corrected.
Restic has also been working great. The problem is that, it has no redundancy to correct errors. The assumption is that, server holding repository will have redundancy, but then I can replicate directly to that server. I worry that at some point, there will be an error in repository. I may loose 5 years of snapshots (restic has some functionality to remove involved snapshots and rebuild the index, but may not succeed).
Kopia has error correction, with Reed Solomon.
Borg2 has perpetually remained in beta. Looking forward to test it.
I stay away from sync: problems with permissions, git repo, conflict, incomplete sync status in syncthing etc.
by aborsy - It sounds like most of your frustration can be eliminated by actually using your NAS as the ground-truth for all data, using something like SMB/NFS instead of SyncThing.
Then you only need to backup the NAS.
by drdexebtjl - I have a server in a closet with a few hard drives attached to it. All of my devices connect to it via Tailscale.
I use the wonderful https://github.com/garethgeorge/backrest as a web UI around restic.
Every night, I back the data up to a Hetzner storage box https://www.hetzner.com/storage/storage-box/ which is only ~$3.50/mo USD for 1TB of data.
I have three "tiers" of data for myself:
1. Important things I care about: these go into a folder that gets backed up to Hetzner (examples include photos in Immich, code pushed to Forgejo, etc)
2. Smaller files I sort-of care about: these usually just go into Proton Drive (examples include design documents)
3. Large files I don't care about losing: these go into a folder on a hard drive that is not backed up (examples include media)
This took me about a day to set up and I just check in on it every once in a while. Once a year I will do a practice disaster recovery run.
- What the author describes here is hard because it's "simple."
What's simple because it's "hard" is replacing parts 2 & 3 with a network appliance like TrueNAS running a zfs pool that syncs to backblaze every night. Yeah you have to learn a bit but it won't fall in weird ways like the hard drive part here will just fail to mount one night and not back things up for 3 months until you notice. My 2¢
by akersten - > One of the other materials that I could not figure out how to back up properly is emails. The reason is that there is no clear way to back it up systematically
Doesn't that depend entirely on the storage format? For example if you use thunderbird as a client it has database files that you can copy. Since you say that one of your devices is running arch you should check out some of the email options in the repos. There are at least a few of options that can act as a middleman to bridge between your external accounts and your clients.
> So having a plan to roll back to a stable state might be helfpul. I installed and set up Timeshift for this
Snapshots make this trivial. Particularly in the case of btrfs if you configure your bootloader to always use the default subvolume then rollback becomes just `btrfs subvolme set-default`.
by fc417fc802 - Some advice from someone who's done it wrong for years and dealt with dead people who have done it wrong for years.
Firstly don't try and run a corporate level backup solution or NAS with terabytes of storage and tape at home. You're going to drop dead one day and everyone will hate you for what you leave them to deal with. They will have no idea how to pick through the remains to get important stuff out. It took me a whole 3 days, as someone who knows how this works, to get into a dead relative's NAS appliance and find any mention of any info to get into his EV which had basically bricked itself in the driveway while he was in hospital.
Secondly, delete everything. Not joking. Delete as much as you can. Everything you've watched, delete it. All the old emails, delete them. Tracks you don't like on CDs you've ripped, delete them. Old software ISOs, delete them. Thousands of photos of the same thing taken 1 second apart, delete them. Thousands of PDFs of books you will never read, delete them. I went down from a whopping 8TB to 300Gb doing that and it's still shrinking monthly. Eventually it'll fit on one mediocre computer and that makes life so so so much easier.
Then look at your backup strategy. It'll look simple then. Mine is three external disks.
1. 1TB T7 shield (not encrypted) which lives at my partner's house and gets an rsync once a month.
2. 1TB T7 shield (encrypted) which lives in my bag and gets an rdiff-backup once a week.
3. 2TB Lacie HDD (not encrypted) which lives in my fire safe at home and gets an rdiff-backup once a week.
If I drop dead, my partner can just plug the thing into her computer and get stuff off it.
I'd avoid the cloud if possible as well. One of the worst situations I've seen is someone who confidently pushed their NAS contents to S3 and then when the NAS blew up they had to pay a lot of money in transfer to get it back again. Hundreds of dollars in fact. On top of the price of a new NAS. That might be a last resort option but it should NEVER be the first line backup.
Don't make your life any more complicated than it needs to be. I implore you.
by sdcfgy - One thing I would add to a modern backup strategy: a deferred offline copy
Sure it's bothersome to use the sneakernet once a month (manually copy data to USB stick and stash it somewhere), but when that ransomware hits (can happen on personal devices also), it literally can't access the offline copy, and try to encrypt that one also.
Immutable backup solutions are code and thus eventually defeatable from the remote attackers point of view. Offline requires physical access, and maybe a big enough wrench to get the USB stick's disk encryption PIN out of you.
by bblb