Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


High-availability file system
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

High-availability file system

MrOwenMrOwen Member
edited October 2012 in General

I may be beating a dead [high availability] horse here, but I just wanted to put this out in the open because I don't ever recall seeing before it on LET (for a server-side solution, at least).

Was looking around a bit for replication solutions for the file system (after @netomx asked for HA solutions) and rsync is really the way to go (industry standard type stuff?) even given some of its shortcomings but I did find one sweet alternative.

I came across a project called SparkleShare. I think this is aimed at a more consumer level, but I see absolutely no reason why it can't be used to sync 2+ servers. It uses git as the backend so file syncing should happen immediately (as opposed to a rsync cron) and with less overhead because it doesn't need as much processing power as rsync.

On top of this amazingness, SparkleShare is a two-way street which means while your main server is down, the backup servers have to ability to bring the main server up to date after it comes back online.

Plus, it seems like it's under very active development.

Comments

  • I think Sparkleshare is really just a beautified version of git, which is not exactly that LEB friendly in terms of memory consumption I believe. Also, why not just use Git itself?

  • @zhuanyi said: I think Sparkleshare is really just a beautified version of git, which is not exactly that LEB friendly in terms of memory consumption I believe. Also, why not just use Git itself?

    This could be true as I haven't installed it on a box yet. I'm doing so right now and I'll report back on the results when I get them.

  • zhuanyizhuanyi Member
    edited October 2012

    @MrOwen said: This could be true as I haven't installed it on a box yet. I'm doing so right now and I'll report back on the results when I get them.

    I have not installed it either but that's the feeling I got after reading their docs. But actually you made a really good point, probably the easiest way is to use one of these 2 way sync software, like Dropbox or git, since you could always launch the daemon say every 5 minutes and sync the contents. For a small website, I think Dropbox would do with 2GB of disk space.

  • joepie91joepie91 Member, Patron Provider
    edited October 2012

    Not a plug-and-play filesystem, but definitely worth a look: https://tahoe-lafs.org/trac/tahoe-lafs

    Redundant and encrypted, and decentralized to a degree.

    Thanked by 1MrOwen
  • MaouniqueMaounique Host Rep, Veteran
    edited October 2012

    @zhuanyi said: lunch the daemon say every 5 minutes

    No offense, just for fun :)
    M

    Thanked by 1zhuanyi
  • @Maounique said: No offense, just for fun :)

    Sorry, it is lunch time here :) Typo corrected.

  • What's the requirement?

    Is the requirement that all files are available regardless of actual uptime of the component nodes?

    Or is the requirement that no data is lost regardless of actual uptime of the component nodes?

  • @Damian said: What's the requirement?

    >

    Is the requirement that all files are available regardless of actual uptime of the component nodes?

    >

    Or is the requirement that no data is lost regardless of actual uptime of the component nodes?

    Well, preferably both but file availability is numero uno priority followed closely by maintaining next to 0 data loss. Obviously, rsync takes care of the first one but not so much the second concern.

  • gbshousegbshouse Member, Host Rep

    You can try XtremFS

  • fileMEDIAfileMEDIA Member
    edited October 2012

    Use Glusterfs, with 4 nodes you can build a ha raid 10 network storage or 2 servers for raid1 storage. We using it for ha vpses based on solusvm.

  • @fileMEDIA said: Use Glusterfs, with 4 nodes you can build a ha raid 10 network storage or 2 servers for raid1 storage.

    Thought about this solution already and it's pretty heavy duty and definitely *not* LEB friendly. Gluster requires a separate volume be mounted plus their recommended hardware is well above most LEB (in terms of memory mostly). Thanks for the suggestion though!

  • fileMEDIAfileMEDIA Member
    edited October 2012

    I think is a easy solution, you can mount a Glusterfs Volumen like a nfs share. You can configure two vserver for a ha storage and all should be fine. You don´t need any big configuration or something, install glusterfs from repo, create a volume and define both nodes. Mount the storage and all works. 10min or so and all should be fine.

    I use a setup with 2 vserver (50GB hdd, 1 core, 1024mb ram) and a load balancer (nginx) in front of the cluster to publish a website.

  • OCFS2 might be of interest.

  • You may want to take a look at OpenStack's storage solution:
    http://www.openstack.org/software/openstack-storage/

  • We're using MooseFS at the office and have had good experiences with it so far. It's less consumer-friendly though, but it's good, stable and has lots of options.

Sign In or Register to comment.