Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Advertise on LowEndTalk.com
Clustering Help?
New on LowEndTalk? Please read our 'Community Rules' by clicking on it in the right menu!

Clustering Help?

UnixfyUnixfy Member
edited December 2016 in Help

Hello LE(S)(B)Tians,

I have a bunch of LowEnd VPSes in several locations scattered throught the US/Europe (only 1 node per location though). I want to make a redundant cluster that is also geographically optimized (i.e node that is closest to a user serves them). I can probably handle the geo part with GeoDNS from Route53, but I'm not sure how to go about making it redundant (I was considering rsync + VirtualMin cluster).

Each node has about the same specs: 1GB RAM, 20GB Disk, 1TB bandwidth.

Thanks!

EDIT: I want to create a cluster of multiple servers, all running a full web stack (LAMP/LEMP).

Visit my website || Add me on Discord: Unixfy#3061

Comments

  • ALinuxNinjaALinuxNinja Member
    edited December 2016

    What are you looking to create a cluster of?

    Webservers or other apps?

    If you are bored and want to try new things for reundancy, setup a consul cluster. Consul clients will check if a particular service is online, and report the status back to the consul server.

    Then, for example if you wanted to route traffic to the closest online server that has a MySQL service on it, you would simply use the hostname mysql-1.service.consul in your apps. This will return a list of hosts that are online and can service MySQL requests.

    You can even seperate consul clients into geographical regions/datacenters (similar to the AWS zones/regions concept) so that the DNS request will choose a client that is in your geographical region/datacenter.

    Devops Consultant | GitHub

  • ALinuxNinja said: Webservers or other apps?

    I was thinking a whole web stack (LEMP or LAMP) because there is enough RAM to run all of that.

    Consul seems pretty interesting, and it has a beautiful UI too...could give that a shot. But the thing that I don't like about it is that it has a one single "master" server, if I understand correctly, once that master server goes down, everything else is gone too?

    Visit my website || Add me on Discord: Unixfy#3061

  • @karatekidmonkey You could get Anycast DNS, and have everything replicate through ex. Syncthing

  • FlamesRunnerFlamesRunner Member
    edited December 2016

    rsync is much easier to do IMO, perhaps that'll work @karatekidmonkey

    wget https://s.flamz.pw/dl/bench.sh && bash bench.sh

    curl https://s.flamz.pw/analytics/bench/stats.php

  • Wouldn't sweat 'clustering' the L, A/E is easily done with rsync, M could be master/master/slaves or a more proper cluster setup in MySQL/MariaDB directly, P.... not sure what a PHP cluster is

  • doghouch said: @karatekidmonkey You could get Anycast DNS, and have everything replicate through ex. Syncthing

    Anycast is too expensive, and the only company that I know does it is BuyVM.

    FlamesRunner said: rsync is much easier to do IMO, perhaps that'll work @karatekidmonkey

    Trying to figure it out right now :)

    alown said: Wouldn't sweat 'clustering' the L, A/E is easily done with rsync, M could be master/master/slaves or a more proper cluster setup in MySQL/MariaDB directly, P.... not sure what a PHP cluster is

    MariaDB cluster? Example/link maybe?

    I don't really need to cluster php because 1) you can't and 2) it's just server software, and the only thing that needs to be synced is the configs, which I can do via rsync

    Visit my website || Add me on Discord: Unixfy#3061

  • @karatekidmonkey Late reply: I believe that Seflow has free Anycast for one domain

    @matteob

    Thanked by 1matteob
  • doghouch said: @karatekidmonkey Late reply: I believe that Seflow has free Anycast for one domain

    I don't want to use Seflow...matteo seems like a jerk.

    Visit my website || Add me on Discord: Unixfy#3061

  • >

    you absolutely can cluster PHP using FPM. :)

    Thanked by 1Unixfy

    No longer with ND

  • matteobmatteob Member, Host Rep
    edited December 2016

    @karatekidmonkey said:

    doghouch said: @karatekidmonkey Late reply: I believe that Seflow has free Anycast for one domain

    I don't want to use Seflow...matteo seems like a jerk.

    Just put in the note field that you not want interact with me and i will be excluded by your tickets.

    Seflow != Matteob

    Matteo Berlonghi - SeFlow s.n.c.
    SeFlow.Net - Affordable DDoS Protected Services. SeFlow Secure Network 20+ IXP connected with Default DDoS Protection.

  • pbgbenpbgben Member, Provider

    @matteob said:

    @karatekidmonkey said:

    doghouch said: @karatekidmonkey Late reply: I believe that Seflow has free Anycast for one domain

    I don't want to use Seflow...matteo seems like a jerk.

    Just put in the note field that you not want interact with me and i will be escluded by your tickets.

    Seflow != Matteob

    Ha, I like this.

  • Haproxy with mariadb and you can give different weight to each server

  • @karatekidmonkey said:

    ALinuxNinja said: Webservers or other apps?

    I was thinking a whole web stack (LEMP or LAMP) because there is enough RAM to run all of that.

    Consul seems pretty interesting, and it has a beautiful UI too...could give that a shot. But the thing that I don't like about it is that it has a one single "master" server, if I understand correctly, once that master server goes down, everything else is gone too?

    Can have multiple master servers.

    Thanked by 1Unixfy

    Devops Consultant | GitHub

  • @matteob said:

    @karatekidmonkey said:

    doghouch said: @karatekidmonkey Late reply: I believe that Seflow has free Anycast for one domain

    I don't want to use Seflow...matteo seems like a jerk.

    Just put in the note field that you not want interact with me and i will be escluded by your tickets.

    Seflow != Matteob

    nvm him; your services are great! :)

    Thanked by 1matteob
  • matteob said: Just put in the note field that you not want interact with me and i will be excluded by your tickets.

    Erm. Okay. Sure.

    What's the difference between l/rsync? Local/Remote?

    doghouch said: nvm him; your services are great! :)

    Also 2expensive4lelcheapome

    Visit my website || Add me on Discord: Unixfy#3061

  • I liked using glusterfs as well. Worked great to sync actual content.

  • matteobmatteob Member, Host Rep

    @doghouch said:

    @matteob said:

    @karatekidmonkey said:

    doghouch said: @karatekidmonkey Late reply: I believe that Seflow has free Anycast for one domain

    I don't want to use Seflow...matteo seems like a jerk.

    Just put in the note field that you not want interact with me and i will be escluded by your tickets.

    Seflow != Matteob

    nvm him; your services are great! :)

    Thank you, glad to hear!

    Matteo Berlonghi - SeFlow s.n.c.
    SeFlow.Net - Affordable DDoS Protected Services. SeFlow Secure Network 20+ IXP connected with Default DDoS Protection.

  • @matteob said:

    doghouch said: @karatekidmonkey Late reply: I believe that Seflow has free Anycast for one domain

    I don't want to use Seflow...matteo seems like a jerk.

    Just put in the note field that you not want interact with me and i will be escluded by your tickets.

    Seflow != Matteob

    nvm him; your services are great! :)

    Thank you, glad to hear!

    I Have tried you service indirectly couple months back its great

    So give LET some time to forget the Awkward moment

    We all will get around I know LET is not you target market but We can make a lot of sound you use it make music or noise (Currently it noise) Will change

    Make your choice on your own But i can help you to make them right.

  • matteobmatteob Member, Host Rep

    @simonindia said:

    Thank you, in last months we hired and started to improve existing service in term of stability and quality instead adding new features. We will start adding it when all bug will be fixed and service rock solid.

    I'm happy to hear that customers noticed that and are happy with the service.

    Thanked by 1Unixfy

    Matteo Berlonghi - SeFlow s.n.c.
    SeFlow.Net - Affordable DDoS Protected Services. SeFlow Secure Network 20+ IXP connected with Default DDoS Protection.

  • @matteob said:

    @simonindia said:

    Thank you, in last months we hired and started to improve existing service in term of stability and quality instead adding new features. We will start adding it when all bug will be fixed and service rock solid.

    I'm happy to hear that customers noticed that and are happy with the service.

    Oh really? Cool, I might give you guys a shot :)

    Visit my website || Add me on Discord: Unixfy#3061

  • rpcoperpcope Member
    edited December 2016

    @ALinuxNinja said:
    What are you looking to create a cluster of?

    Webservers or other apps?

    If you are bored and want to try new things for reundancy, setup a consul cluster. Consul clients will check if a particular service is online, and report the status back to the consul server.

    Then, for example if you wanted to route traffic to the closest online server that has a MySQL service on it, you would simply use the hostname mysql-1.service.consul in your apps. This will return a list of hosts that are online and can service MySQL requests.

    You can even seperate consul clients into geographical regions/datacenters (similar to the AWS zones/regions concept) so that the DNS request will choose a client that is in your geographical region/datacenter.

    The only thing to note is that Consul doesn't really play nice if you've got geographically separated server agents in the same Consul "datacenter", as by default Consul is pretty aggressive about failing nodes. Additionally, it's unclear how much thought and hardening around security have gone into it's development, and I think if I were going to run Consul on systems on the open web, I'd either lock access down in iptables to only systems I knew needed to talk, or run a VPN for all of the connections between nodes. Otherwise, it's a pretty neat and useful tool.

    Also, no if you're running with only one server for Consul, you're doing it wrong if it's not just for simple development purposes. Production deployments usually consist of at least 3 (hopefully more) "server" nodes.

    Thanked by 1Unixfy
Sign In or Register to comment.