Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Is 50% high for a promox server during backup?
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Is 50% high for a promox server during backup?

The title says it all. I was only backing up a single VM to a backupsy server over a freelan peer to peer tunnel using NFS. During this it said the IO wait was about 50%, and that was when the interface would load at all. It had a difficult time bringing up information, like trying to get it to bring up information on the current backup job was impossible.

It only has a single drive with two quad core processor and plenty of RAM. Is having that single disk really causing that to be so high?

Comments

  • Well, I think it'd depend on the amount of files and their sizes etc...

    From what I've seen, running rsync on a single 50 GB File does not really spike the I/O or anything, but when I'm doing it for a 50 GB folder that has 50,000 1MB Files, that's a little problematic. However, 50% is a little bit too high of the backup is the only process running.

  • ZEROFZEROF Member
    edited October 2014

    I will agree with @DalekOfSkaro, just if you have folders with 10000 files it's better to set pre-backup files compression and then run backup.

  • @DalekOfSkaro

    That's strange then, I'm pretty sure it compresses the file before sending it over. And backing up the KVM machine it only does a couple of files. The biggest being the hard disk image.

  • How are you backing up? Rsync over NFS over tunnel?

    What is the tunnel type? MTU/MSS/etc? Are you getting resends? Fragmentation?

  • @MarkTuner

    It's whatever proxmox uses to backup, I think it uses rsync but I'm not sure. For the tunnel it's the default MTU of 1500 and its a SSL tunnel using this software. The documentation needs a little work but so far it was able to create a SSL tunnel between my two servers. Not sure if its the best thing to use or not, but I needed to be able to mount a NFS share to proxmox.

    How would I check for MSS, resends, and Fragmentation?

  • @CFarence - are you backing up OpenVZ or KVM instances?

    Why are you backing up over NFS. NFS + WAN is not a good combination. NFS is not a happy bunny with latency, fragmentation and reduced MTUs.

    Are you backing up from a Delimiter server?

  • @MarkTurner

    Yeah its from a Delimiter server, I'm using proxmox and I'm pretty happy with it though it only gives you a couple options to add storage NFS or GlusterFS. Not really familiar with GlusterFS. I don't know if I could mount the remote storage some other way and then add it to proxmox as a directory?

  • But which virtualisor are you using? OpenVZ or KVM?

    Do you want offsite storage or just any backup? There are options available at Delimiter for backup and you can get the storage delivered on a private network over ISCSI or NFS.

    I wouldn't run NFS over a WAN, its just a recipe for extreme pain. In Proxmox you could backup to local disk and then rsync the files over to your off-site backup.

    Either set the backups for midnight and then rsync cronjob to go out at say 0200. If you use rsync over ssh then you don't need an additional encryption layer to keep the majority from viewing your data; and you don't need the NFS pain or tunnel to move the data.

  • CFarenceCFarence Member
    edited October 2014

    @MarkTurner
    I have both OpenVZ and KVM machines, I have a VPS with 512MB ram and a 500GB drive for 7$/month so that's my offsite storage. I'm going to look into setting up the local backup and then rsync job. It sounds a lot more efficient that way, and if I want to restore a VM from a backup I don't need to transfer it back over the internet.

  • It will also avoid the NFS/tunnel pain.

    You may also be able to hook Proxmox's backup job so when it completes it passes the filename of the backup over to a script to rsync it.

  • Thanks @MarkTurner I'm going to drop the NFS/tunnel thing and look into rsync jobs.

  • Ok if you need help, then let me know. I did write a tutorial on this once but can't find it right this second

Sign In or Register to comment.