I have a server running Debian with 24 TB of storage. I would ideally like to back up all of it, though much of it is torrents, so only the ones with low seeders really need backed up. I know about the 321 rule but it sounds like it would be expensive. What do you do for backups? Also if anyone uses tape drives for backups I am kinda curious about that potentially for offsite backups in a safe deposit box or something.

TLDR: title.

Edit: You have mentioned borg and rsync, and while borg looks good, I want to go with rsync as it seems to be more actively maintained. I would like to also have my backups encrypted, but rsync doesn’t seem to have that built in. Does anyone know what to do for encrypted backups?

  • lemmyvore@feddit.nlEnglish
    0·
    3 months ago

    As long as you understand that simply syncing files does not protect against accidental or malicious data loss like incremental backups do.

    I also hope you’re not using --delete because I’ve heard plenty of horror stories about the source dir becoming unmounted and rsync happily erasing everything on the target.

    I used to use rsync for years, thinking just like you, that having plain old files beats having them in fancy obscure formats. I’m switching to Borg nowadays btw, but that’s my choice, you gotta make yours.

    rsync can work incrementally, it just takes a bit more fiddling. Here’s what I did. First of all, no automatic --delete. I did run it every once in a while but only manually. The sync setup was:

    • Nightly sync source into nightly dir.
    • Weekly sync nightly dir into weekly dir.
    • Monthly tarball the weekly dir into monthly dir.

    It’s not bad but limited in certain ways, and of course you need lots of space for backups — or you have to pick and choose what you backup.

    Borg can’t really get around the space for backups requirement, but it’s always incremental and between compression and deduplication can save you a ton of space.

    Borg also has built-in backup checking and recovery parity which rsync doesn’t, you’d have to figure out your own manual solution like par2 checksums (and those take up space too).

    • bandwidthcrisis@lemmy.worldEnglish
      0·
      3 months ago

      Re needing lots of space: you can use --link-dest to make a new directory with hard links to unchanged files in a previous backup. So you end up with de-duplicated incremental backups. But borg handles all that transparently, with rsync you need to carefully plan relative target directory paths to get it to work correctly.

    • ancoraunamoka@lemmy.dbzer0.comEnglish
      0·
      3 months ago

      As long as you understand that simply syncing files does not protect against accidental or malicious data loss like incremental backups do.

      Can you show me a scenario? I don’t understand how incremental backups cover malicious data loss cases