Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


vps backup (rsync / tar /...) and restore from bare metal/vanilla state
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

vps backup (rsync / tar /...) and restore from bare metal/vanilla state

r0nr0n Member

What method's/strategies can be used (or do you use) to FULLY backup a VPS (ubuntu)?

What I tried:

TAR:

tar -cvpzf /restore/backup1.tar.gz --exclude=/restore --exclude=/proc --exclude=/lost+found --exclude=/sys --exclude=/mnt --exclude=/media --exclude=/dev /
Here you see I excluded some directories.

RSYNC:

  • backup:
    rsync -ra --delete --progress --delete-excluded --exclude 'proc/' --exclude 'restore/' / /restore

  • restore:
    rsync -ra --delete --progress --exclude 'restore/' /restore/ /

  • Optional, tar the "restore" folder:
    tar -cvpzf /restore/rsync-backup.tar.gz --exclude=/restore/rsync-backup.tar.gz /restore

Here the "restore" folder is located in the root and I rsync almost the complete root into the "restore" folder. The rsync command could use some more exclusions(?)

TAR ERROR:
tar exiting with failure status due to previous errors

RSYNC ERROR:
some files/attrs were not transfered...

The rsync could use some more exclusions, but I wanted to know if there are other/better methods to do a full backup (and restore). Or do I need to adjust my commands?
I used the above commands years ago (somewhere in 2011) and they seemed to work then (ubuntu) I recently am trying to pick up my VPS learning. I had a server and domain running on the VPS back then the backing up and restoring then restarting the VPS seemed to work back then.

Ubuntu root folder descriptions
http://www.makeuseof.com/tag/folders-linux-root-directory/

If someone could guide me into the right direction or could provide a complete command example, I would really appreciate it. I would prefer to keep it within the RSYNC / TAR realms if possible.

Thanks.

Comments

  • Use something like vzdump on the host node if possible. Or if you want to back up the whole fs, use dump/restor. These days I have a fairly consistent set of packages installed, so I just back up my home dir with tar/rsync/duplicity/whatever. Then I can wipe out the vps, reinstall from scratch, and re-initialize it from an ansible script.

    That has the benefit of getting rid of accumulated cruft and breakage in the system, i.e. if I "customized" something and forgot about it, I probably want to get rid of it instead of backing it up and restoring.

    Thanked by 1r0n
  • you can ommit the r in -ra of your rsync command as it is included with the flag a. you could instead think about adding AX to preserve more of the attributes if needed/applicable. I'd also exclude /dev and /sys from the backup and would backup to another server rather then the system itself.

    to do differential backups and have the option to restore multiple different dates way back have a look into rdiff-backup which is based on rsync...

    Thanked by 1r0n
  • @Falzo said:
    I'd also exclude /dev and /sys from the backup and would backup to another server rather then the system itself.

    Good hint.

    to do differential backups and have the option to restore multiple different dates way back have a look into rdiff-backup which is based on rsync...

    And another good point.

    Thanked by 1r0n
  • dump/restor is historically how you do it to rebuild a bare computer after a hard drive fails among other things. So it gets all the special files in /dev etc. For vps backup maybe you want something different.

    Thanked by 1r0n
  • Let us not forget the effort each approach requires. To backup /dev, for instance, will rarely be worth the effort; a fresh install will be simpler and cleaner.

    Based on what little the OP has said, IMO Falzo has pretty much nailed it down.

    So, let me bring up another issue that is rarely discussed: Think about a reasonable file structure and keep it in order, i.e. differentiate between what the system needs and does and what you, the user, needs and does in terms of data.

    One example regarding the system is to regularly dump the list of all installed packages (e.g. on a debian system dpkg --get-selections > /home/sysbackup/installed.lst). This also allows to recreate the current package situation if a new setup was required.

    It might also be worth to think about why certain tools (e.g. rsync) have been created in the first place and what your situation is. With the first part I mean that many tools are decades old; the world was different then and some of the problem of then simply aren't problems anymore nowadays. With the second part I mean that the situation - and hence the choice of tools - is quite different for a company server, for a hoster (with hundred or more of systems to backup) or for a private user with a VPS for a couple a web sites.

    Thanked by 3r0n user123 nessie
  • This is a long post to enable understanding so that you can use this to suit your own needs - it is not a shortcut to just cut-paste. I hope Cloudflare doesn't eat my (long) post. Otherwise I will post in bits.

    I use rsync (extensively) to backup (virtual) machines to avoid the hassle of a fresh-install-reconfiguration (since I have many things setup/tuned on the machine). I use these backups to create fresh instances (quite regularly) for certain "classes" of machines. Due to the way I have things setup, the only difference between A and B (same class) is the IP address - everything else is "generated" from the IP (hostname, machine specific configs like ssh host keys etc.). Note that these machines are otherwise "identical" (have the same software setup to do the same things). The key differences are things like logs and possibly other data files (specific to the machine) which are either start-from-empty or else restore-from-somewhere-else type of things.

    I use Debian (8) and let udev manage /dev (to save me the hassles) and they work for OpenVZ/KVM/Xen (the differences being in the "class" of the machine and some related files specific to the class either symlinked differently or else present/absent as the need may be).

    Here's how I take a backup:

    /usr/bin/rsync -aHXSA / /backups/CURHOST/ \ --exclude /sys/** --exclude /proc/** --exclude /run/** \ --exclude /var/log/** --exclude /tmp/** \ --exclude /backups \ -hh --stats --numeric-ids

    (command broken over many lines with continuations to help readability)

    (the numeric-ids isn't necessary on the same machine but doesn't hurt)

    Add/Customize the excludes based on your own specific setups (maybe /opt and/or /data etc.). Ensure that you either exclude the /etc/network/interfaces and /etc/fstab files on restoring to a DIFFERENTLY configured machine.

    The above copies the data to the same machine but it is easy and straightforward to change the target (to either push from the CURHOST to BACKUPHOST or pull from BACKUPHOST to CURHOST).

    Now your source backup is ready.

    To restore on the SAME configuration machine (including partitions if Xen/KVM/Hardware), you can clobber everything via flipping the source and destinations on the above rsync command. Otherwise exclude /etc/network/interfaces and /etc/fstab on restore.

    If you are restoring from a rescue medium (Xen/KVM/HW) you can do this to the mounted (destination partition) folder without any issues (as nothing is running from there). On OpenVZ you need to ensure that you are running only the bare minimum services (sshd, rsyslog/etc, udev/systemd-udev etc.) Shutdown everything else including cron.

    Once the restore is completed (note that it would have excluded /var/log/ files etc.) clean out what you don't want based on your excludes.

    You're technically ready to reboot. Check that all is well and you can try shutdown via
    shutdown -r -n now (a normal reboot will probably fail but shouldn't matter).

    After the boot, your "new" (old) machine is running and all should be well.

    I use a similar (manually invoke script) method to spin up my instances (VPS and others) from a template. My main change is that I "rebuild" the host configuration files prior to the reboot based on the hostname/IP address.

    This doesn't go into details like LVM setups, Grub reinstall etc. which are a little more involved but follow the same principle with a few extra (manual, straightforward )steps thrown in between.

    I've mentioned similar points on some previous threads related to setting up new VPSs etc.

    Hope this helps.

    (END OF POST)

    Thanked by 2r0n user123
  • A few more additions (thanks to Cloudflare)

    ETC slash HOSTS will also need special treatment depending on if it will be auto generated (common with OpenVZ) vs self managed (for other types of machines).

    P.S: Why does CF not like certain words?

    Thanked by 1r0n
  • WSSWSS Member

    dd if=/dev/sda of=$(netcat pipe)

    Thanked by 1r0n
  • r0nr0n Member

    Wow guys! although the thanks-button was made for this I still wanted to thank you for all the info, it really opened up a world for me!

Sign In or Register to comment.