New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Well, duplicity sucks. Any other backup recs?
raindog308
Administrator, Veteran
in General
Here are some typical results of running backups with duplicity on something as wild and exotic as Debian 8:
Traceback (most recent call last):
File "/usr/bin/duplicity", line 1509, in
with_tempdir(main)
File "/usr/bin/duplicity", line 1503, in with_tempdir
fn()
File "/usr/bin/duplicity", line 1352, in main
do_backup(action)
File "/usr/bin/duplicity", line 1473, in do_backup
full_backup(col_stats)
File "/usr/bin/duplicity", line 545, in full_backup
globals.backend)
File "/usr/bin/duplicity", line 427, in write_multivol
(tdp, dest_filename, vol_num)))
File "/usr/lib/python2.7/dist-packages/duplicity/asyncscheduler.py", line 145, in schedule_task
return self.__run_synchronously(fn, params)
File "/usr/lib/python2.7/dist-packages/duplicity/asyncscheduler.py", line 171, in __run_synchronously
ret = fn(*params)
File "/usr/bin/duplicity", line 426, in
async_waiters.append(io_scheduler.schedule_task(lambda tdp, dest_filename, vol_num: put(tdp, dest_filename, vol_num),
File "/usr/bin/duplicity", line 315, in put
backend.put(tdp, dest_filename)
File "/usr/lib/python2.7/dist-packages/duplicity/backends/_ssh_pexpect.py", line 226, in put
self.put_sftp(source_path, remote_filename = remote_filename)
File "/usr/lib/python2.7/dist-packages/duplicity/backends/_ssh_pexpect.py", line 239, in put_sftp
self.run_sftp_command(commandline, commands)
File "/usr/lib/python2.7/dist-packages/duplicity/backends/_ssh_pexpect.py", line 192, in run_sftp_command
child.sendline(self.password)
File "/usr/lib/python2.7/dist-packages/duplicity/pexpect.py", line 961, in sendline
n = self.send(s)
File "/usr/lib/python2.7/dist-packages/duplicity/pexpect.py", line 953, in send
c = os.write(self.child_fd, s)
TypeError: must be string or buffer, not None
Traceback (most recent call last):
File "/usr/bin/duplicity", line 1509, in
with_tempdir(main)
File "/usr/bin/duplicity", line 1503, in with_tempdir
fn()
File "/usr/bin/duplicity", line 1352, in main
do_backup(action)
File "/usr/bin/duplicity", line 1441, in do_backup
list_current(col_stats)
File "/usr/bin/duplicity", line 674, in list_current
sig_chain = col_stats.get_signature_chain_at_time(time)
File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line 977, in get_signature_chain_at_time
raise CollectionsError("No signature chains found")
CollectionsError: No signature chains found
My opinion is that anything that results in an error might be on me, but something that results in the program dumping a traceback is on duplicity...I know it's old but it seems half-baked.
Yeah, i'm using pexpect for sftp because paramiko support lacks support for some ssh options.
I guess I need to write my own tar-encrypt-and-sftp script again...will miss incrementals, though. Anything better out there?
Comments
borgbackup? I haven't used duplicity enough to compare with it, but borg seems like the Valhalla of backups no matter your use case
Wow, that looks very interesting...even discusses append-only mode to prevent a hacked client from removing backups. Impressive! reading docs now...
Definitely give borgbackup a try. I've been using it and have nothing to complain about really (except a few nags like not trying to compress some standard extensions that are already compressed/wont compress much).
I also like the fact that there's a "big huge binary" available for many platforms/distributions which is very convenient without having to install too much stuff and can help you get going quickly with the binary literally just after downloading.
I've been using Duplicati 2.0 for a while now (https://www.duplicati.com/), on a Windows machine though, but it should support Debian also.
Same as @Aleksio. Using Duplicati for my important servers to backup up to Amazon Cloud. Using caddy as a reverse proxy to protect the interface.
Did.
No support for sftp.
Became sad.
Looked at Duplicati...I guess they're just too cool for docs.
"Getting started guide...This guide assumes you have already installed Duplicati". OK then. From what I could tell from the articles - which are on random topics? - I need a web server running on the client..? It looks kind of half-baked...
If I recall correctly it's an all-in package, no need for a separate webserver, it's built in.
That just seems very heavy weight for a 128MB VPN box...
Well I guess I can get what I want from borg...maybe. I have to run borg serve, which I suppose isn't the end of the world. I prefer working over ssh with accounts limited to internal-sftp and chrooted, with nologin for a shell...well, there will be some fiddling.
yes, ssh authorized_keys limited to running only 'borg serve'.
Yeah, duplicati definitely isnt for a 128mb vps.
What speaks against rsync?
Plot twist
What do you want to save from 128MB VPN box?
Looking rclone since some days ago.
Maybe that combined with the mounted backups feature and sftp/sshfs or something like that and a simple bash script may work for you?
Hum, that seems odd, it does work on jessie for me, not in 128 MB though.
Try duply and see if there is an issue with the executables or the commands somehow.
rclone with ACD and on-the-fly encrypt is PURE GREATTTTTTT!!! works in terminal (ssh) etc, folder mount, upload to mounted dir etc etc ... so easy & unlimited space
This was on a 1G box actually.
well, well...
Time Machine -> ENcrypted volume on AFP -> ZFS -> Snapshots
Ah, Linux... rsync.
@raindog https://github.com/restic/restic
Chiefly that I'm backing up, not synchronizing.
i.e., there's no versioned backups in rsync. Also no full vs. incr. Also no encryption. Also...
I've been using borg backup extensively. Works solidly. It can also (fuse)mount backups so you can access them as a normal file system, but it takes quite a bit of memory when there are more backups involved! So then pruning becomes important.
https://github.com/restic/restic/issues/299
https://github.com/restic/restic/issues/299#issuecomment-277457578
https://github.com/restic/restic/issues/21
https://github.com/restic/restic/issues/407
Droping an eye at Borg. Interesting stuff. Thanks for sharing with the community.
You might want to try http://rdiff-backup.nongnu.org
rsnapshot perhaps?
Duplicity is not sucks
Did you try rsnapshot? Never tried it, but it's on my to-do list...
I think rsnapshot will also have the same SFTP issue that @raindog308 has with borgbackup. I use both (different uses) but of course I have ssh access/shell on the required machines.
Both are great in their own ways though borgbackup will be much lighter on disk due to the deduplication and fewer files/inodes (but as a result it is much higher on CPU/RAM).
Pick any two : disk space or CPU or Memory :-)
Decided to try again because the duplicity list said my version was old.
Upgraded to 0.7.11 which meant downloading and building myself, distributing, etc.
First backup? Another Traceback.
True, I'd included a 0-byte exclude file and that confused it, but that's kind of the point. Only poor programmers accept input without testing it. duplicity is written by poor programmers apparently.
It's also a very wonky program because it uses either paramiko or pexpect. paramiko is buggy garbage and also refuses to accept many ssh options, unfortunately. pexpect does accept this, but you have to install that apt and then be careful you use the right magic incantation - e.g., -oPort doesn't work, but :port does, etc. It takes a very careful reading of the man page.
As I said, duplicity sucks. Unfortunately, there may not be another option. I want to like borg, but its lack of sftp support is a real pain.
With sftp, I can
With borg I can chroot if I have a separate standalone server which...look, I'm sure they're sincere, but they're not openssh.
I used to tar/compress/gpg myself and effectively do fulls every night...I may go back to that :-(
@raindog308 - did @vimalware's suggestion of running borg serve via authorized_keys not work out?
rsync works nicely via authorized_keys (and I suppose borg will too) - I'm still a bit confused (for lack of a better term) on what the concern is in allowing borg to run similarly. I'm sure you can lock arguments down so that it is essentially a backup only process (i.e. no "read/restore" access to the repo). (and no I'm not saying it's as GOOD or trustworthy as ssh but....)
I confess I didn't look at it deeply, but my impression was that
Also, it meant building a chroot environment for the user, which is a maintenance headache (.e.g, you have to have a bin, an etc, etc. in the user's chroot, and then either you have 50 copies of that for every user, or one shared environment, or a lot of symlink trickery).
I'd like users to only be able to login, chrooted to their backup directory, and only be able to add/remove files in that directory (remove only because duplicity, etc. need to maintain, though I should see if that can be done server side).
I should probably play with it more.