Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Good disaster recovery plan for web server
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Good disaster recovery plan for web server

vanarpvanarp Member
edited March 2013 in Help

I run a web server (LEMP stack using Tuxlite script) on a LEB. I would like to have a disaster recovery plan should something happen to my VPS. Appreciate all suggestions and pointers in this regard.

«1

Comments

  • I wrote a really small script which backups my mysql database + forum files every day to Dropbox, that's what i'm using

  • erhwegesrgsrerhwegesrgsr Member
    edited March 2013

    Mirroring

  • NeoNeo Banned

    Rsync daily with Mysql Backup?

  • Off-site R1Soft backups

  • jhjh Member

    R1soft

  • raindog308raindog308 Administrator, Veteran

    Mirroring/rsync is fine...until your site is hacked and you find your mirror has a copy of the hack.

    r1soft is pretty ex$pensive last time I looked. "what's your data worth" blah blah...but the OP has a leb .

  • dmmcintyre3dmmcintyre3 Member
    edited March 2013

    If run daily, this will give you the last 10 days of backups to restore from if something bad (hacking/etc) happens.

    #!/bin/bash
    cd /var/backup
    rm backup.10.tgz -rf
    mv backup.9.tgz backup.10.tgz
    mv backup.8.tgz backup.9.tgz
    mv backup.7.tgz backup.8.tgz
    mv backup.6.tgz backup.7.tgz
    mv backup.5.tgz backup.6.tgz
    mv backup.4.tgz backup.5.tgz
    mv backup.3.tgz backup.4.tgz
    mv backup.2.tgz backup.3.tgz
    mv backup.1.tgz backup.2.tgz
    tar cvpf backup.1.tgz files mail -X /var/backup/excludes
    rsync --delete -avz -e "ssh" user@hostname1:/var/www/ /var/backup/files/hostname1/www
    rsync --delete -avz -e "ssh" user@hostname2:/var/www/ /var/backup/files/hostname2/www
    rsync --delete -avz -e "ssh" user@mx:/var/mail/ /var/backup/mail

    Then use http://sourceforge.net/projects/automysqlbackup/ for MySQL.

  • I'm using duplicity for easy website and database backups: https://raymii.org/s/tutorials/Website-and-database-backup-with-Duplicity.html

    also includes mysql and mongo backup instructions.

    Run it daily or hourly, when something goes wrong you reinstall, tuxlite, duplicity restore and done.

  • raindog308raindog308 Administrator, Veteran

    Another option is to upload to Amazon Glacier. One penny per GB per month.

  • jhjh Member

    @raindog308 said: "what's your data worth" blah blah...but the OP has a leb .

    $150 to own it for life, or roughly $7-10/month to rent it. Seems quite reasonable for what it is if your data is actually worth something.

  • @raindog308 I love s3cmd for s3 sync/backup but what do you use for glacier?

  • @jhadley said: roughly $7-10/month to rent it. Seems quite reasonable for what it is if your data is actually worth something.

    VM Agents can go as low as $3/mo in v5.x ;)

  • AaronAaron Member

    @raindog308 I love s3cmd for s3 sync/backup but what do you use for glacier?

    glaciercmd, of course!

    https://github.com/LordGaav/glaciercmd

  • @Aaron saw the java requirement and ignored it previously lol

  • raindog308raindog308 Administrator, Veteran
    edited March 2013

    @Aaron said: glaciercmd, of course!

    It's java...I haven't tried it but I assume it's not leb-friendly.

    It's pretty straightforward to write something with python. python-pip install boto, which gives you access to a rich library of AWS APIs. Here is a tutorial I found on using glacier with it:

    http://thomassileo.com/blog/2012/10/24/getting-started-with-boto-and-glacier/

    @jhadley said: $150 to own it for life, or roughly $7-10/month to rent it. Seems quite reasonable for what it is if your data is actually worth something.

    I think I'm not understanding something...the web site says $1000.

    http://r1soft.idera.com/pricing

  • @raindog308 said: I think I'm not understanding something...the web site says $1000.

    http://r1soft.idera.com/pricing

    Thats for either 50x VM agents or 5x Dedicated server agents.
    If you go through a R1Soft partner (btw we are one ;) then you don't have to pay a one-off fee, you can go monthly + at a lot cheaper rates.

  • KuJoeKuJoe Member, Host Rep

    In addition to backups, this is the setup we use for our main website (SD.net):

    2 varnish load balancers in different locations.
    Round-robin DNS for both load balancers.
    2 webservers in different locations behind the load balancers.
    WebserverA does an rsync every minute to WebserverB (static website but replicated MySQL is extremely simple to setup if needed).

    If both webservers go down, the varnish will display the cached pages. I recommend getting a DDOS protected VPS for one of the load balancers for best uptime.

    Of course, you can always change the number of webservers and load balancers to meet your needs.

  • bnmklbnmkl Member
    edited March 2013

    @KuJoe, your attention to uptime could part the thighs of any woman. Perhaps even buttocks.

  • raindog308raindog308 Administrator, Veteran

    @KuJoe said: I recommend getting a DDOS protected VPS for one of the load balancers for best uptime.

    Gee, I wonder if there is a provider here who sells something like that...:-)

  • KuJoeKuJoe Member, Host Rep

    @raindog308 said: Gee, I wonder if there is a provider here who sells something like that...:-)

    We're not in the LEB price range anymore. :( I was using BuyVM for one of our load balancers for a while though. :)

  • KuJoeKuJoe Member, Host Rep

    @bnmkl said: @KuJoe, your attention to uptime could part the thighs of any woman. Perhaps even buttocks.

    It's easy to do when it doesn't cost anything. :)

  • AdducAdduc Member

    rsync non-image files and use version control to track changes to these files. I implemented this with personal projects, and convinced my employer to beginning using this method early 2012 with great success. We had a wordpress site set up using this method, and was able to identify a hack due to an outdated plugin within minutes of it occurring.

  • Thank you all for the great suggestions.

    I do not want to spend money on external services at this time. Instead prefer to make use of my LEBs. So the option to sync everything from one vps to another (in addition to regular backups) seems like an optimal and cost-effective solution. The idea is to have least (if not zero) down time before the sites are available.

    Wondering if there is any script that someone has already coded to handle rsync of files/directories and MySQL replication in regular intervals. Most of the websites are WP driven.

    @KuJoe said: Round-robin DNS for both load balancers.

    Assuming I have two web servers with same files and configs, how to enable this round-robin DNS? Will it cause any SEO issues with Google (two IPs showing same content)?

  • raindog308raindog308 Administrator, Veteran
    edited March 2013

    @KuJoe said: 2 varnish load balancers in different locations.

    Round-robin DNS for both load balancers.

    I'm curious what's meant by "load balancers" here.

    So my browser queries DNS and gets the IP for node1 as the IP for www.example.com. Seconds later node1 crashes. I load another page for www.example.com and...it won't know that node2's IP exists and it's not going to query again (for some period of time).

    Even with a low TTL, most browsers are going to ignore it and some upstream DNS won't honor anything below an hour.

  • raindog308raindog308 Administrator, Veteran

    Really...that's cool.

  • lzplzp Member

    every day

    Wow, you guys don't care much about your data.

    I back up my database every hour, two hours and day. All three are necessary in case I need to revert to a slightly older version.

    I back up all of my files (including the database dumps) with rsync every hour. Since rsync will only download new files, there is no bandwidth lost - only better backups gained.

    I guess if the only thing you have is static websites that you rarely update, or blogs that you never update, it might not be that important.

  • @bdtech thanks for the link.

  • tchentchen Member

    @vanarp csync2, lsyncd, or even unison are good options for the file sync that work fine on a LEB. I'm partial towards csync2 given its low overhead.

    As for MySQL, just set the backup server as a slave and be prepared to promote it. Everyone and their sister has written and put something up on github to wrap that process to make it friendlier - although I behoove you to actually do it manually a few times so that you know the basic commands required. And you'd probably need mysql proxy unless you want to edit the mysql connection strings during the switchover too.

    An alternative is to just use galera cluster (or percona xtradb cluster) and really not worry about that whole switchover business.

  • GIANT_CRABGIANT_CRAB Member
    edited March 2013

    Cloud, SAN, multiple cloud fail over, RAID 1010, file backups to cloud, roundrobin load balancing, IP failover, DNS clustering, anycast CDN and DDoS protection.

Sign In or Register to comment.