Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Change disk on OpenVZ
New on LowEndTalk? Please Register and read our Community Rules.

Change disk on OpenVZ

DrukpaDrukpa Member

I have a dedicated server with OpenVZ (Centos 5) running few VMs. Currently has a 2x500GB SATA in SW RAID1. Have been noticing a decrease in performance with IO wait times going very high (~30%), causing server loads to increase (upto 10) for a combined total of ~5% of the day. Rest of the times, the loads stays below 1.

The processor is a Intel Xeon e3 1230 with 16GB memory.

Am thinking of replacing the SATA drives with a 2x240 SSD in SW RAID1 if its not too much work.

I am a developer and not a server admin, so my knowledge of server management is just basic/intermediate. Can take care of daily problems on my own by reading tutorials, etc., but am thinking changing disks and migrating data is no small stuff with a lot of risk for data loss. Ofcourse I do have keep backups on a different server.

So what could be the easiest way to achieve what I want?

Replace disk, migrate everything with the minimal amount of down time.
I don't want to setup a different server with the SSDs, setup everything and then move data from the old server. Doing that would have to change my IP address and stuff.

Comments

  • wychwych Member

    Is the server colo'd or rented?

    It may be easier biting the bullet for a month and getting an additional box, that way you can just migrate them.

    Taking a hiatus.

  • linuxthefishlinuxthefish Member
    edited April 2015

    Check smart data for the drives first, I see a lot of drives that go mega slow before actually failing!

    smartctl -a /dev/sda, then smartctl -a /dev/sdb and look for any reallocated sectors or any failed values.

    Easiest way would be to use a tool such as vzdump, or even vzmigrate to a temporary node if you have ploop would be quite fast.

    You could also just backup /vz/private and /etc/vz with permissions if you are feeling brave...

  • @wych said:
    Is the server colo'd or rented?

    It may be easier biting the bullet for a month and getting an additional box, that way you can just migrate them.

    Server is rented. I believe getting a new server would mean getting a new range of IPs. I host my own DNS (cpanel) and have about 400 domains pointed to the old IP. Am not sure it would automatically change those DNS entries, but if it doesn't, am in for some pain. So I really need the same IPs.

    @linuxthefish said:
    Check smart data for the drives first, I see a lot of drives that go mega slow before actually failing!

    smartctl -a /dev/sda, then smartctl -a /dev/sdb and look for any reallocated sectors or any failed values.

    Easiest way would be to use a tool such as vzdump, or even vzmigrate to a temporary node if you have ploop would be quite fast.

    You could also just backup /vz/private and /etc/vz with permissions if you are feeling brave...

    Doesn't show any reallocated errors, but it does say "OLD AGE".

    Do you think if I do a vzdump to the backup server, change disks, setup the node (this time with centos 6 oepnvz) and import the old dump, it will work? Don't mind a few hours down time.

  • wychwych Member

    @Drukpa Ask your host to move the IP's to the new server.

    Taking a hiatus.

  • DrukpaDrukpa Member
    edited April 2015

    This is my IO test result :D

    dd if=/dev/zero of=iotest bs=64k count=16k conv=fdatasync
    16384+0 records in
    16384+0 records out
    1073741824 bytes (1.1 GB) copied, 158.086 seconds, 6.8 MB/s
    
  • Does your current server have the space to add the two SSD's along side the current disks for a period of time?

  • @AshleyUk said:
    Does your current server have the space to add the two SSD's along side the current disks for a period of time?

    Yes, I can use max 4 disks. So 2 is free at the moment.

  • MikePTMikePT Member, Provider

    @Drukpa said:
    This is my IO test result :D

    dd if=/dev/zero of=iotest bs=64k count=16k conv=fdatasync
    16384+0 records in
    16384+0 records out
    1073741824 bytes (1.1 GB) copied, 158.086 seconds, 6.8 MB/s
    

    That is horrible! :P

  • SadySady Member

    Your best bet would be to vzdump all of the containers, move those to somewhere else (Something in same DC will be better to have good network speeds), swap HDDs with SSDs & then simply grab those dumps back & restore those.

    I don't have any relation/affiliation with any LET Host, All of my comments are my own
    Simple bash script to clean compromised wordpress site [cPanel/WHM specified]

  • @Sady said:
    Your best bet would be to vzdump all of the containers, move those to somewhere else (Something in same DC will be better to have good network speeds), swap HDDs with SSDs & then simply grab those dumps back & restore those.

    Is vzdump reliable? Any chance the dumps become corrupted/un-restorable?

  • @Drukpa said:
    Yes, I can use max 4 disks. So 2 is free at the moment.

    Depending on the downtime you can suffer, could do a offline disk migration, using a bootable iso/disk.

    This would allow you to just make a straight copy of your current system to the SSD's.

    Or you could install a fresh OS to the second set of disks in Raid 1 and then extract the VZ data folder from the old disks.

    Depends how much downtime your looking for, if not vzdump else where swap disks and reinstall OS and then move the VZ dumps back.

    No way is going to be without some form of downtime.

  • Or just get new HDDs that are not from the 90's!

  • Buy new server.

    Migrate all containers

    Port forward old IPs

    change DNS. This time use cname

    Different.

  • Drukpa said: Intel Xeon e3 1230 with 16GB memory.

    And the disks are from the days of the PC XT :)

    Out of interest which provider is this?

    If you have RAID1 then you could fail one disk, install a replacement wait for it to replicate the data then fail the other disk and repeat.

  • @MarkTurner said:

    Can't do that if the new disks are smaller than the old ones....

  • @sleddog - clearly not, but the key is to get off the breaking/dying disks

  • I don't see evidence the disks are dying. Poor dd could well be VM users hammering them. Which I guess amounts to the same thing :)

  • komputerkingkomputerking Member, Provider

    I recently just performed a migration similar to what you are asking. Ask the datacenter if they can build a replacement server in the same vlan, so that you can share ips, and just migrate them all to your new setup. Once all of them are transferred, ask the DC to transfer the ips, and dismantle the old server.

    With it being in the same vlan, it'll be a quick transfer, and none of your clients will be required to change ips. They'll just have a 10-15 minute downtime as the data is transferred, give or take.

  • komputerkingkomputerking Member, Provider

    Another option you have, is making use of the ftp backup in solusvm, if you are using that. You can deploy a new server, use this scheduling setup so that it backs up and transfers the data to your new server, and then just bring them up. Make use of the vzmigrate command...

    /scripts/vm-migrate [VSERVERID] [NEWNODEID]

    to update solusvm that they have been moved, once you restore them, and delete them from the original server.

Sign In or Register to comment.