Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Beta testers for hosted Borg backup services wanted. 100GB free space
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Beta testers for hosted Borg backup services wanted. 100GB free space

m4num4nu Member, Patron Provider

I'm a big fan of Borgbackup and couldn't find a good hosting service for it. So I built one that I'd want to use last week. It's great to back up very cheap VPS servers that may fail at some point. Hehe.

Some notable features I wanted and already added:

  • monitoring to find failed backup scripts.
  • append-only repositories to be safe against someone hacking a server and removing the backup
  • keep every backup repo separate instead of having a single account, like many providers have.
  • choose EU or US location

Currently I'm doing a public beta which will last until mid-2019. If you know or use Borg or look to use it, I'd be happy to sign you up as beta user.

Link: https://www.borgbase.com/

«13456

Comments

  • donlidonli Member
    edited October 2018

    Is the borg going to assimilate my data?

    Fully Encrypted with Your Key

    Please tell us more about the encryption.

    Any idea yet as to the pricing of the product?

  • @donli said:
    Is the borg going to assimilate my data?

    Fully Encrypted with Your Key

    Please tell us more about the encryption.

    https://borgbackup.readthedocs.io/en/stable/faq.html#security

  • m4num4nu Member, Patron Provider

    Thanks for adding the link. Basically you initialize the repo on your local machine and set a key at the same time. Borg will only upload compressed/deduplicated/encrypted segments.

    Borg supports different hashing modes, which are all hardware accelerated on most CPUs. Overview is given here. Currently repokey-blake2 or keyfile-blake2 are recommended by the lead maintainer. (he helped me improve the configuration wizard).

    In addition to encryption you can enable append-only mode which stops Borg from ever cleaning old files and allows to go back to a previous transaction. E.g. if a hacker deletes your snapshots, you can simply undo those changes later. This is the main benefit of my service compared to e.g. rsync.net.

    Regarding pricing it will be in line with the rest of the industry. Plus you only pay for compressed and deduplicated data usage. So you use much less than with other services for many types of data.

  • So where can I find the best tutorial on say, how to Borg content of a Centos VPS to your service?

  • m4num4nu Member, Patron Provider

    Good question. I noticed that some new users never added a SSH key or repos. I probably have to link to more resources. There are some instructions under "Setup". Anyways, here are some rough steps. You still need basic command line experience:

    1. Install Borgbackup. CentOS has a package for it. So it's probably just yum install borgbackup for you. Full guide here

    2. Add SSH key on borgbase.com under Account/SSH. I recommend making a new key on the machine that will use it. So ssh-keygen and then cat ~/.ssh/id_rsa.pub. Then just paste that public key in the key data field.

    3. Add repo on borgbase.com. The easy part. Just New Repo. Select a name, the space you need and the SSH key. Optionally you can choose EU or US location.

    4. Head to the Setup page on borgbase.com. There you can find the command to init the repo and make your first backup. After that I'd look at the official docs to see what else borg does.

    5. Operationalize. When you're ready to switch to regular backups, most people use Borgmatic or Backupninja. I provide a config template for Borgmatic under Setup. You just need to adjust the folders you want to back up.

  • Signed up; I can always use another bit-o-borg. Cheers!

  • m4num4nu Member, Patron Provider

    Great. Sent the next batch of invites just now. Any bugs you encounter or suggestions, just let me know by replying to the welcome-email.

    Thanked by 1CyberMonday
  • Presignuptual: Your privacy link should probably oldschool target="somewhereelse", and clicking back completely nukes any previous data entered as is the newschool ideology, but it is still annoying.

  • m4num4nu Member, Patron Provider

    True. That's not great. I'll change that.

  • rsync.net also loves Borg. Good luck with your project!

  • m4num4nu Member, Patron Provider

    Right. rsync.net were pioneers. They are just not specialized in Borg only and don't offer features I was missing.

    • keep all repos separate.
    • only access repos via Borg to get the benefits of append-only.
    • monitoring to get alerted of stale backups.
    • choose the borg server version to use to avoid broken backups (got this feedback from current rsync.net users)
    • multiple locations to be GDPR compliant.

    Basically I'm going one level up to offer additional features.

  • Sounds like a great project!

    You may want to have a look at Hetzner. They are also investing on Borg as a backup solution to their Storage Boxes.

  • What are you guys used for mysql backup?
    automysqlbackup is not ready for MySQL 5.7

  • m4num4nu Member, Patron Provider

    I actually have 2TB with Hetzner's Borg offering right now. It was terrible enough to make me take a shot at the problem myself. Problems are similar as rsync.net. In detail:

    • everything under one user. My backups are not separated. I can't put multiple clients or servers in the same account. Can't limit the quota for individual servers or clients. One rogue server could fill up the whole account.
    • The web interface is confusing and they ask you to enter/trust them with your repo key. Why?
    • They use a non-standard port for borg. So you need to adjust that in every tool.
    • No real append-only mode. The client can always log in via SFTP and do whatever they like with your Borg backups.

    My plan is basically to offer better tooling around Borg by not doing anything else (SFTP) at the same per-GB price as the current providers who offer Borg as afterthought to their current service. Any feature requests and ideas to improve it are still welcome.

    Thanked by 2beagle vimalware
  • @rsync_dot_net is on LET but not so active.

    @m4nu said: everything under one user.

    I thought that this was a feature, not a bug. (But I see your point.)

  • m4num4nu Member, Patron Provider

    (Bumping up) Still have a few TB of beta space to give out.

  • I am not interested myself, but wanted to say that it's a nice project. Good luck!

    Thanked by 1m4nu
  • m4num4nu Member, Patron Provider

    Thanks man. One of my first own startups after usually working for other people.

    Thanked by 1Shazan
  • dragon2611dragon2611 Member
    edited October 2018

    @m4nu said:

    • keep every backup repo separate instead of having a single account, like many providers have.
    • choose EU or US location

    There are both advantages and dis-advantages to this.

    It's better for secuirty but you can get better de-duplication if multiple servers are accessing the same repo, that said Borg doesn't support concurrency so only one could backup at time.

    Restic is also an interesting backup tool ,although I don't think it does as much in terms of compression as borg so the backups end up bigger (But 2 servers can backup to the repo at the same time).

    That said they are both great tools and anything that encourages people to backup more often and to encrypt their offsite backups is a good thing in my opinion.

  • m4num4nu Member, Patron Provider

    dragon2611 said: It's better for secuirty but you can get better de-duplication if multiple servers are accessing the same repo, that said Borg doesn't support concurrency so only one could backup at time.

    In most use cases I know of, the data on machines is unrelated. But if you like, you can still back up multiple servers to the same repo.

    Restic is working on compression. It's also nice to have so many different backends available. Of course this shifts more responsibility to the client and it's harder to implement features like monitoring and append-only. Different tools for different use cases.

  • How does Borg deal with "bit rot" when data is encrypted?

  • @m4nu This seems a very good project! Keep up.

    Thanked by 1m4nu
  • m4num4nu Member, Patron Provider

    Excellent question. There is a discussion about this on Github. The short summary being that Borg has mechanisms to detect bit rot but not to correct it.

    As a user you can use the borg check command to make sure the checksums of all backup segments match. This would be similar to ZFS checksums. It's also the pre-defined standard in the Borgmatic template you get from the the BorgBase.com Setup Assistant. From the docs:

    For all objects stored in the segments, all metadata (e.g. crc and size) and all data is read. The read data is checked by size and CRC. Bit rot and other types of accidental damage can be detected this way.

    If any damage should be detected, Borg can try to recover files from the damaged segment. In addition you can use the --verify-data to look at individual backed up files. Takes longer, but more thorough.

    The --verify-data option will perform a full integrity verification (as opposed to checking the CRC32 of the segment) of data, which means reading the data from the repository, decrypting and decompressing it. This is a cryptographic verification, which will detect (accidental) corruption.

    There seems to be some controversy on the topic and whether file systems can be used to improve the situation. I'm still considering whether it makes sense as a hosting provider to use ZFS. I know rsync.net does. But they back up individual files and for them ZFS is the only defense against bit rot. In the case of BorgBase, Borg has built-in bit rot detection that has a large overlap with ZFS. Until I hear new information, my recommendation would be:

    • use Borg's built-in check feature during every backup (as is the default with BorgBase's Borgmatic config template)
    • repair the backup using the original data in the unlikely event of remote bit rot.
    • the possibility of having massive hardware bit rot locally AND remotely at the same time for the same files is very very small.
    • for very important data, back up to 2 repos. This is also very easy to set up with Borgmatic.
    • Also: BorgBase is built as operational backup solution for data that is in active use and has a copy elsewhere. It's not built as long-term archive solution. For this I'd recommend B2 or (maybe) AWS Glacier.

    If anyone with experience has input on this, I'd love to hear it. Especially whether ZFS adds much benefit for storing Borg repos.

  • m4num4nu Member, Patron Provider

    Got some good feedback to display SSH host keys for validation. Now implemented and online:

  • m4num4nu Member, Patron Provider

    Thanks again for testing everyone. There is now a macOS/Linux desktop client available for BorgBase. Actually it's fully open source and you can use it with any SSH account you want, not just BorgBase. Just a way to make it easier to use on desktops.

    There is still work to be done, but the current version should be fairly usable. Looking forward to any improvement ideas and bug reports on Github.

    If you need advanced features, like monitoring, separated repos and append-only mode, I'll be happy if you try out BorgBase. But you can also connect it to any SSH box that has the borg-binary available.

    Latest release is on Github under "Releases". Currently only packaged for macOS, but Linux will follow.

    Thanked by 2beagle adamluk
  • m4num4nu Member, Patron Provider

    Got many requests to add 2FA to protect backup repo settings. So here it is. Available to test on https://borgbase.com

    Just enable under Account > Password/2FA. Screenshot.

    If you have bugs or suggestions, just ping me here, on Twitter or via email.

    Thanked by 1Aluminat
  • Great work! Vorta already works very well, I'd love to see it handle local backups, too.

  • m4num4nu Member, Patron Provider

    Local Vorta backups are coming soon. I use them myself as well. The relevant Github issues are:

    Thanked by 1Aluminat
  • bjobjo Member
    edited November 2018

    Gna, bought a liteserver openvz box for borg already and saw your project now.

    Thanked by 1LiteServer
  • m4num4nu Member, Patron Provider

    For how long did you pay in advance? If you only use it for Borg, then BorgBase may be quicker. I used storage VPS as well, but it took too long to set up new backup users (even with Ansible) and monitor for broken backups.

    Since BorgBase is free for now, just use both. And let me know if you need more space. I'll up it for you.

    Thanked by 1bjo
Sign In or Register to comment.