Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In with OpenID
Advertise on LowEndTalk.com

In this Discussion

What do you use to handling automated backup of multiple servers?
New on LowEndTalk? Please read our 'Community Rules' by clicking on it in the right menu!

What do you use to handling automated backup of multiple servers?

Currently my backup solution is rsnapshot. Rsnapshot has served my purpose well, but it have not been updated for some time, so I'm looking for another solution to replace it.

  • Incremental/delta snapshot backup is required.
  • Dedup is optional but nice to have.
  • All of my servers are Linux, and I will not be having windows servers, so there's no need to worry about that.
  • Will be nice to have a centralized web control/status panel.
  • Looking for more of a free/open source solution, though I might consider commercialized if it isn't too expensive.

Recommendations are appreciated, thanks! For peeps using bareos, how is it?

Comments

  • @mgilang said: cdp.me ?

    I've actually checked it out, the last update was Oct 2015 though. How was your experience with it?

  • TommyCTommyC Member

    I use amanda.

    Although I don't know of any web interfaces that it has (if any).

    "I'm just here for the cheap offers."

  • @alexyaoyang said: Currently my backup solution is rsnapshot. Rsnapshot has served my purpose well, but it have not been updated for some time, so I'm looking for another solution to replace it.

    Why replace something that is working fine for you? or are you experiencing issues with it?

  • @TommyC said: I use amanda.

    Although I don't know of any web interfaces that it has (if any).

    Hmm okay I will check Amanda out, thanks!

  • @zafouhar said:

    @alexyaoyang said: Currently my backup solution is rsnapshot. Rsnapshot has served my purpose well, but it have not been updated for some time, so I'm looking for another solution to replace it.

    Why replace something that is working fine for you? or are you experiencing issues with it?

    That's a good question, well since it is not getting updated, as OS/other software gets updated it might start breaking, security concerns, etc. So I decided that now's a good time to start looking for alternatives.

  • jadenjaden Member

    I use duplicity for historical backups (to a Google Cloud Nearline bucket, GPG encrypted) and rclone for mirroring (also to a Nearline bucket).

    One negative is there's no overall backup status or dashboard, but it's cheap and it works.

  • imokimok Member
    edited August 12

    BackupPC, backup and restore files from a SpeedyKVM VPS. Works fine.

  • seanhoseanho Member
    edited August 12

    Lots of folks here using borg. I use burp since I need good support for both Linux and Windows clients.

    [edit: added links]

  • Check BackupNinja

    Thanked by 1szarka
  • alexyaoyangalexyaoyang Member
    edited August 12

    @jaden said: I use duplicity for historical backups (to a Google Cloud Nearline bucket, GPG encrypted) and rclone for mirroring (also to a Nearline bucket).

    One negative is there's no overall backup status or dashboard, but it's cheap and it works.

    @imok said: BackupPC, backup and restore files from a SpeedyKVM VPS. Works fine.

    @seanho said: Lots of folks here using borg. I use burp since I need good support for both Linux and Windows clients.

    [edit: added links]

    @nqservices said: Check BackupNinja

    Thanks for all the suggestions, will check them out over the weekends!

  • alexyaoyang said: I've actually checked it out, the last update was Oct 2015 though. How was your experience with it? used cdp.me last year, it works well, but just for short project.

  • ZerpyZerpy Member

    @mgilang said: cdp.me ?

    You mean the software that exposes a json file to the public interwebs with your server passwords in cleartext?

    Suureee

    Thanked by 1Tom
  • Because Im currently also looking into which software to use: Anybody has any impressions of UrBackup?

  • MagicalTrainMagicalTrain Member
    edited August 13

    Because Im currently also looking into which software to use: Anybody has any impressions of UrBackup?

    Edit: Still online over mobile connection... Guess it posted twice.

  • I looked for a backup system for a while, for 20-30 servers.

    I ended up going with Duplicati (the 2.0 Beta). It doesn't have centralized config, but I install it via Anisble.

    I push a little script with all the details (backup location, password, email settings, etc) and then a cronjob to run at night.

    It does encryption, incremental, supports a bunch of locations (gdrive, s3, b2, etc).

    It installs a neat little web interface if you don't want to do a CLI backup, and restores were really cool, you can pick specific files from specific days and restore them from any machine running duplicati. I'm very happy with it.

    Thanked by 1szarka
  • seanhoseanho Member

    @MagicalTrain said: Because Im currently also looking into which software to use: Anybody has any impressions of UrBackup?

    I trialled urbackup a number of years ago, after giving up on BackupPC and before going with burp. I don't remember what turned me off of it, maybe it had to do with Windows support. I can look up my old notes if you like.

  • @Seanho currently looking into either burp (with burp-ui), backuppc, urbackup or elkarbackup.

    If its too much of a bother, you dont need to specifically rummage through your old notes. Ill probably end up installing test nodes of each anyway (unless somebody has big concerns with any of them).

  • @MagicalTrain said: @Seanho currently looking into either ... urbackup or elkarbackup.

    Both look good. elkarbackup web interface looks really great! Anyone using any of them?

  • seanhoseanho Member

    @MagicalTrain said: @Seanho currently looking into either burp (with burp-ui), backuppc, urbackup or elkarbackup.

    If its too much of a bother, you dont need to specifically rummage through your old notes. Ill probably end up installing test nodes of each anyway (unless somebody has big concerns with any of them).

    That's a prudent plan of action. Backups are important; take your time and choose something you could settle with for the long-term. And by all means, test out a full bare-metal restore!

    Just to note, burp-ui is under active development and still a bit buggy (but I still use it). I would recommend burp2 format for the block-level dedup. BackupPC is old and written in Perl; whether that is good or bad depends on your point of view....

    But: you say all your clients are Linux; I don't see why you wouldn't go with one of the more highly recommended Linux options like duplicati or borg. I haven't extensively tested borg, but from what people say about it, it sounds like a better feature set than burp (apart from good Windows support).

  • MagicalTrainMagicalTrain Member
    edited August 13

    @Seanho

    I do generally like duplicati, but for my backup solution I want one central server from which to control my backups from.(pull vs push basically) With possible duplication to a secondary backup server, but that if a software that does the first one really well, I can take care of the second part via scripts and cronjobs. (as it would only be "1" server to care of after all)

  • @MagicalTrain said: @Seanho With possible duplication to a secondary backup server...

    https://github.com/elkarbackup/elkarbackup/wiki/RedundantBackup

  • seanhoseanho Member

    @MagicalTrain said: @Seanho

    I do generally like duplicati, but for my backup solution I want one central server from which to control my backups from.(pull vs push basically) With possible duplication to a secondary backup server, but that if a software that does the first one really well, I can take care of the second part via scripts and cronjobs. (as it would only be "1" server to care of after all)

    Yeah, I hear you on the central trusted backup server; that's how I'm using burp right now. I can queue backups from the server, and they get run the next time the client checks in. There is something to be said for the untrusted server model, though -- no single point of failure.

  • MagicalTrainMagicalTrain Member
    edited August 13

    @seanho said:

    @MagicalTrain said: @Seanho

    I do generally like duplicati, but for my backup solution I want one central server from which to control my backups from.(pull vs push basically) With possible duplication to a secondary backup server, but that if a software that does the first one really well, I can take care of the second part via scripts and cronjobs. (as it would only be "1" server to care of after all)

    Yeah, I hear you on the central trusted backup server; that's how I'm using burp right now. I can queue backups from the server, and they get run the next time the client checks in. There is something to be said for the untrusted server model, though -- no single point of failure.

    I understand that, but, imo, backups are enough of a pain in the ass that I want to at least give me this little bit of comfort. Having a secondary replicated backup server should take care of any freaky freakouts that may happen.

    @nqservices yeah, I saw that. elkar is currently the first one Im testing. The default install isnt all too secure though.

  • Really surprised no one mentioned https://syncthing.net

    I use it for 1 VPS, you could use it for as many as you want. I set it up to back up 1 VPS to home Windows PC but have run it previously with 1 VPS to 1 backup VPS.

    Thanked by 1szarka
  • @sidewinder said: Really surprised no one mentioned https://syncthing.net

    I use it for 1 VPS, you could use it for as many as you want. I set it up to back up 1 VPS to home Windows PC but have run it previously with 1 VPS to 1 backup VPS.

    WOW THANKS, this reminds me of the old bittorrent sync,

    I was using bittorrent sync to have a mirror of a small vps to my home pc,

    even used it to sync three dns servers..

    going to have fun with this..

  • szarkaszarka Member

    @sidewinder said: Really surprised no one mentioned https://syncthing.net

    Looks interesting—especially because it also supports Android. I actually had this in my bookmarks, but hadn't found time to play with it. The GUI is optional and you can do everything with config files and shell commands, right?

  • @szarka said:

    @sidewinder said: Really surprised no one mentioned https://syncthing.net

    Looks interesting—especially because it also supports Android. I actually had this in my bookmarks, but hadn't found time to play with it. The GUI is optional and you can do everything with config files and shell commands, right?

    I am pretty certain this true, but not sure.

    I heard about it here in these forums and there was a decent thread about surprised no one mentioned it.

    ElkarBackup looks good but I think Synching is in active development is battle tested

  • SplitIceSplitIce Member, Provider

    After not finding anything we liked we actually built our own system. In our case it's an ansible role for configuring backup, an ansible deployed backup server and a monitoring agent that sends out once a day a backup summary.

    Makes it easy to add backups to a server, which in turn helps ensure that every server gets backups made.

    X4B - DDoS Protection: EU & US affordable DDoS protection including Layer 7 mitigation.
    Latest Offer: $7 Anycast DDoS Protection
  • @sidewinder said: Really surprised no one mentioned https://syncthing.net

    I use it for 1 VPS, you could use it for as many as you want. I set it up to back up 1 VPS to home Windows PC but have run it previously with 1 VPS to 1 backup VPS.

    what happen if someone delete files from 'master' or you want to restore let say data from 3 month ago?

    Last time I use synthing it doesn't support file history or incremental backup

  • seanhoseanho Member
    edited August 13

    @sibaper said:

    @sidewinder said: Really surprised no one mentioned https://syncthing.net

    what happen if someone delete files from 'master' or you want to restore let say data from 3 month ago?

    Last time I use synthing it doesn't support file history or incremental backup

    This. Syncthing is awesome, but sync and backup are two different use-cases. It's why rsync is not a backup system on its own, and why rsnapshot, attic, borg, etc came about.

    @SplitIce, I use ansible too, to deploy the backup software, but what do you use to do the actual backups? Did you write your own rsync scripts?

  • SplitIceSplitIce Member, Provider

    Data is trasnferred using sshfs (some of our backups are over 100GB in size and we don't have the temporary space for doing it all locally, and we need to be able to do incremental backups).

    For mysql we do innobackupex For filesystem we use rsync to a sshfs directory

    X4B - DDoS Protection: EU & US affordable DDoS protection including Layer 7 mitigation.
    Latest Offer: $7 Anycast DDoS Protection
  • If you have a service so bad you need backups, why even bother backing up then?

  • @cheapwebdev

    I guess we know why youre cheap now.

    Thanked by 2alown rocket
  • @sibaper said:

    @sidewinder said: Really surprised no one mentioned https://syncthing.net

    I use it for 1 VPS, you could use it for as many as you want. I set it up to back up 1 VPS to home Windows PC but have run it previously with 1 VPS to 1 backup VPS.

    what happen if someone delete files from 'master' or you want to restore let say data from 3 month ago?

    Last time I use synthing it doesn't support file history or incremental backup

    https://docs.syncthing.net/users/versioning.html

    Never tried it, but I'd probably bet it works as advertised.

    So deleted files are recoverable. The name synching is a title misleading when it can do versioning and recovering deleted files.

  • sidewinder said: Never tried it, but I'd probably bet it works as advertised.

    Yes, it works as advertised but only for latest version since that script copy every folder to trashcan folder. If your folder had 10 GB your trashcan had 10 GB too which is not limited storage friendly.

    So it's not fully capable to do backup jobs (backup, deduplication, versioning, etc)

  • lurchlurch Member

    @cheapwebdev said: If you have a service so bad you need backups, why even bother backing up then?

    That is...........hilarious

Sign In or Register to comment.