Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Beta testers for hosted Borg backup services wanted. 100GB free space - Page 5
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Beta testers for hosted Borg backup services wanted. 100GB free space

1235

Comments

  • m4num4nu Member, Patron Provider

    sin said: Are the free beta plans still going on or did those end? I haven't set anything up yet but I'm really loving the layout of the site and this looks like it will be really great to back up all my vms and dedis .

    The beta plan closed to new users on Dec 31. Sorry. :‑[

    Also: just fixing a friend's WP site. No usable backup. Ouch. Always do your backups, kids!

  • @m4nu said:

    Bochi said: Would be nice to have some kind of protection against (accidental) deletion of a repo from within the backup, i.e. a "lock" you could apply which then requires an additional step to delete the repository.

    Can be added. I'm thinking of having users type the repo name, similar to Github. I added an issue for it on Fider.

    Bochi said: Would you mind sharing a few details about the setup in the background?

    The frontend is in Vue.js, which connects to a Django backend via GraphQL (can be used by anyone to automate backup setup). The actual data is kept on separate servers in the respective EU/US locations. Currently a Python daemon is managing repos there, but I'll soon replace that with Golang for a slightly leaner setup and simpler deployment.

    Not a fan of having to type shit out to confirm something. I prefer big, bold scary wording with double check box confirmations instead. I know it's less safe, but when my Paragon software wants me to type hard drive name to delete a partition, it bugs me so much. Even though I know they are trying to prevent bad mistakes. Not sure if this is the ultimate laziness or not.

  • m4num4nu Member, Patron Provider

    TimboJones said: Not a fan of having to type shit out to confirm something.

    Same. How about a slider? You slide it all the way to the right and only then the Delete button gets enabled. Prevents accidential clicking and gives enough time to read the repo name again.

  • @m4nu said:

    TimboJones said: Not a fan of having to type shit out to confirm something.

    Same. How about a slider? You slide it all the way to the right and only then the Delete button gets enabled. Prevents accidential clicking and gives enough time to read the repo name again.

    I think I'd be good with that. Points if you make it look like you're sliding it into the trash.

  • m4num4nu Member, Patron Provider
    edited March 2019

    TimboJones said: I think I'd be good with that. Points if you make it look like you're sliding it into the trash.

    Implemented this. You move the slider into the trash can. Not sure this is cool enough for points.

    There are also some improvements to SSH key management. Like seeing the date it was added and auto-filling the key name. (Turns out some people are too lazy to make up a key name.)

    image

  • Any redundancy in the backend like aws s3 (something like 3x replication)? And would be nice if there was a software for windows with gui :smile:

  • m4num4nu Member, Patron Provider

    @sanvit said:
    Any redundancy in the backend like aws s3 (something like 3x replication)? And would be nice if there was a software for windows with gui :smile:

    Actually I have good news on both things:

    • Windows support for Borg is being worked on and we may see it later this year. Vorta can already run on Windows, but Borg is missing a few pieces.

    • Currently I have an offsite backup of all user data in addition to local RAID. So your backup would even survive a DC outage. In the future I'll probably use CephFS to store user backups. It's more complicated to run, but I'm not aware of other sane ways to grow large storage pools.

    I won't be able to have 3x replication at the current price, but I plan on using erasure code to account for 2-3 HD failures in a given cluster. The benefit of EC vs. replication is that it only needs ~120% of disk space vs 300% for replication.

    (This is something I'm still working on/testing. If anyone has experience in running a petabyte Ceph cluster, I'd love to hear their experience.)

    If you're worried about data loss, I also encourage you to back up your data to both the EU and US region. Borgmatic can already back up to multiple repos in a single run.

  • @m4nu said:

    @sanvit said:
    Any redundancy in the backend like aws s3 (something like 3x replication)? And would be nice if there was a software for windows with gui :smile:

    Actually I have good news on both things:

    • Windows support for Borg is being worked on and we may see it later this year. Vorta can already run on Windows, but Borg is missing a few pieces.

    • Currently I have an offsite backup of all user data in addition to local RAID. So your backup would even survive a DC outage. In the future I'll probably use CephFS to store user backups. It's more complicated to run, but I'm not aware of other sane ways to grow large storage pools.

    I won't be able to have 3x replication at the current price, but I plan on using erasure code to account for 2-3 HD failures in a given cluster. The benefit of EC vs. replication is that it only needs ~120% of disk space vs 300% for replication.

    (This is something I'm still working on/testing. If anyone has experience in running a petabyte Ceph cluster, I'd love to hear their experience.)

    If you're worried about data loss, I also encourage you to back up your data to both the EU and US region. Borgmatic can already back up to multiple repos in a single run.

    Thanks for the clarification! The 3x replication was just an example, and I also think this is not doable with your pricing (or even for more)! Would be happy to try out when the windows software rolls out. Good luck with your backup! :)

  • m4num4nu Member, Patron Provider

    Anyone feeling adventurous today? The first version of a Ceph-backed cluster for storing Borg backups is now online. You can test it by choosing the "EU-Ceph" region when creating the repo.

    image

    This region is only for testing and there is only single parity (can't lose more than 1 drive).

    Using Ceph probably doesn't provide many advantages to users, but it's great for providers (which many of you guys are). What I'm looking forward to:

    • add or remove servers from the cluster without worrying about the actual files
    • faster rebuilds after drive failures (large RAID drives can take days to rebuild and you can't lose more drives while it's happening)
    • ability to access the same FS from multiple places and scale horizontally

    Anyone else has some experience with Ceph or any gotchas to share?

    Thanked by 1Aluminat
  • I don't have experience setting up Ceph but a very bad experience like many here with providers using Ceph (ZXHost anyone?).

    It does seem to work great until it doesn't. And then there is no way back from that.

    I hope it works better for you.

    Thanked by 1vimalware
  • @beagle said:
    I don't have experience setting up Ceph but a very bad experience like many here with providers using Ceph (ZXHost anyone?).

    It does seem to work great until it doesn't. And then there is no way back from that.

    I hope it works better for you.

    Wasn't ZXHost's problem a financial one, and not a technical one?

  • @sanvit said:

    @beagle said:
    I don't have experience setting up Ceph but a very bad experience like many here with providers using Ceph (ZXHost anyone?).

    It does seem to work great until it doesn't. And then there is no way back from that.

    I hope it works better for you.

    Wasn't ZXHost's problem a financial one, and not a technical one?

    If I'm not mistaken, it all started with their Ceph backed storage VPSs going to s**t because of a bug on Ceph. Many users paid for 3 yrs plans so they tried to rent new servers and setup a new infra-structure using RAID to offer these customers. Since they had already paid for the Ceph infra they were trying to salvage and having now to pay for the new and more expensive infra they ran out of money.

    If you are interested you can have a look at this thread:

    https://www.lowendtalk.com/discussion/129635/do-you-have-a-problem-with-zxhost

    Thanked by 2m4nu sanvit
  • m4num4nu Member, Patron Provider

    beagle said: It does seem to work great until it doesn't. And then there is no way back from that.

    From what I learnt so far, there are 101 ways to go to hell in handbasket with Ceph. Some configurations can be risky, if you don't know what you're doing. Hence some testing.

    On top, I keep some daily cold backups of everything. Just in case.

  • m4num4nu Member, Patron Provider

    beagle said: Since they had already paid for the Ceph infra they were trying to salvage and having now to pay for the new and more expensive infra they ran out of money.

    Just glanced over the thread. Thanks for this cautionary tale. I do wonder why they didn't simply repave their cluster instead of going a route that wasn't viable financially.

    Often (business) people expect too much of some over-hyped tech and quickly hit a wall. Maybe that's what happened.

    Thanked by 1beagle
  • nqservicesnqservices Member
    edited April 2019

    Hey @m4nu Can you please add on your KB/Docs the following 2 articles:

    1- How to download the backups made to our own computer.
    2- How to restore the backup to the same server backups were made.

    Thanks

  • FYI, I have a Nextcloud server that is 140GB due to trashed version files adding up, but borg repo is like 32GB. Really impressed by that.

    Thanked by 2Aluminat bjo
  • m4num4nu Member, Patron Provider

    nqservices said:
    1- How to download the backups made to our own computer.
    2- How to restore the backup to the same server backups were made.

    Sure. I didn't write a new article in a while anyways. Any specific OS or application you want to see?

    TimboJones said: FYI, I have a Nextcloud server that is 140GB due to trashed version files adding up, but borg repo is like 32GB. Really impressed by that.

    Good to hear. Probably a lot of deduplication going on. Borg will chunk files up and only store new chunks. If only a small part of the file changed, only the change gets stored. Compression helps a bit too. Especially Facebook's ZStd has a nice speed/size ratio.

  • nqservicesnqservices Member
    edited April 2019

    @m4nu said:

    nqservices said:
    1- How to download the backups made to our own computer.
    2- How to restore the backup to the same server backups were made.

    Sure. I didn't write a new article in a while anyways. Any specific OS or application you want to see?

    1- How to download the backups made to our own computer.

    How to download backups at BorgBase to a Windows 10 computer.

    2- How to restore the backup to the same server backups were made.

    In my case Debian and Ubuntu is the case.

    Note 1: I know it takes time, but when you can it would be great to have some video screencast tutorials. I rally like your service and I think having more easy and clear tutorials wll help you gain much more clients.

    Note 2: One thing I notice is that I only find the Docs/KB link after login. You should put a public link on your website so new clients/visitors can read them before sign-up. If you already have the public link sorry but I did not see.

    Thanks! Please continue with your great service!

  • When I say:

    1- How to download the backups made to our own computer.

    How to download backups at BorgBase to a Windows 10 computer.

    I mean how to download the backups / repo to our own local computer. I said Windows 10, but that is not important. I mean how to download the repo in a ZIP or TAR file for example. So the client can use a Windows, Max or Linux computer. It should be the same process since the download will be made using browser.

  • @nqservices said:
    When I say:

    1- How to download the backups made to our own computer.

    How to download backups at BorgBase to a Windows 10 computer.

    I mean how to download the backups / repo to our own local computer. I said Windows 10, but that is not important. I mean how to download the repo in a ZIP or TAR file for example. So the client can use a Windows, Max or Linux computer. It should be the same process since the download will be made using browser.

    No, there is no downloading from borgbase gui. You need a command line client that has repo key.

  • @TimboJones said:

    No, there is no downloading from borgbase gui. You need a command line client that has repo key.

    Be able to download from GUI would be a great feature! Anyway for now simple step-by-step KB/Docs guide on how to do it is needed.

  • m4num4nu Member, Patron Provider
    edited April 2019

    I won't support downloading unencrypted files from the web interface because this would mean I need the encryption key. Not a good idea.

    BUT you can already download files using the Vorta desktop GUI. Just add an extra SSH key to your BorgBase server repo and you're all set.

    image

    If you just need a few files and want to browse multiple snapshots you can also use the Mount feature.

    image

    Just keep in mind that Mount has some overhead from FUSE. So Extract is better for more data.

  • @m4nu said:
    I won't support downloading unencrypted files from the web interface because this would mean I need the encryption key. Not a good idea.

    Ok, I understand. Security first!

    BUT you can already download files using the Vorta desktop GUI. Just add an extra SSH key to your BorgBase server repo and you're all set.

    Went check Vorta but it's only available for Linux and Mac, correct? Any GUI for Windows 10?

    Thanks

  • m4num4nu Member, Patron Provider

    Borg already works well using Windows 10's Linux Subsystem.

    Native Windows support is under way. From what I know the main issue is just testing and packaging. There shouldn't be large technical challenges.

    Vorta is based on Qt and can run on Windows without issue. We're just waiting for Borg to officially support Windows. Maybe someone here has some experience and can package it for Windows?

  • roastroast Member

    count me in if it's still available

  • @m4nu said:
    Borg already works well using Windows 10's Linux Subsystem.

    Thanks for all the info. When you can and have the time please write the following tutorials on your website DOCS/KB section:

    1- How to download the backups made to our own computer.
    2- How to restore the backup using SSH to the same or other Debian/Ubuntu server.

    Thanks!

  • bjobjo Member

    @m4nu Did you disable the ceph-repos? I'm getting Permission denied (publickey)since yesterday.

    Thanked by 1m4nu
  • m4num4nu Member, Patron Provider

    I had to repave the cluster yesterday and didn't set everything up until about 2h ago. Should work again. Else drop me an email with the ID. I copied the old data over.

  • james50ajames50a Member
    edited April 2019

    -

  • m4num4nu Member, Patron Provider

    nqservices said: 1- How to download the backups made to our own computer.
    2- How to restore the backup using SSH to the same or other Debian/Ubuntu server.

    Added an article that covers restoring backups using either borg mount or borg extract.

    Hope it's useful. The install commands are probably a bit rough and I didn't test them everywhere. So it's likely not 100% copy&paste.

Sign In or Register to comment.