New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Comments
are they connected via private network?
there are a lot of options like nfs, glusterfs, zfs, sshfs and so on, but depends on many things from the given environment which could be the best...
Nah Public Network.
glusterfs/sshfs can be used safely over the internet as long as the latency isn't too high...
GlusterFS doesn't use an encrypted connection by default, though. It does support SSL, but there is next to no documentation on how to configure it to use it.
sshfs looks nice for 2 Dedis,but i think when i mount the directory on two sides i get problems so far.
@Infinity580, don't get me wrong: it's entirely possible to set up encryption for GlusterFS (if you use a recent enough version); it's just annoying to find and apply what little documentation there is about it.
SSHFS is much simpler to set up, of course, but it only makes remote storage available locally; you'll have to set up synchronisation of local and remote storage yourself.
What do you mean by that?
I think he means sshfs allows a local system to access a remote system but doesn't do any synchronisation/distributed storage setup etc.
and there is that file attributes thing too.
GlusterFS works the best for my use, if in the same dc, or close by on 1GB I see 30MB or so transfer rates.
maybe @Infinty580 would like to outline a bit more on what he's planning to do between those two servers ;-)
AFAIR I'd guess he wants to combine disk space from both and share this space on both? therefore I would go with glusterFS.
only if it would be enough mounting some directories vice versa to gain access, I would go with sshfs for the easy of use.
How about Section 2 of
https://www.debian-administration.org/article/438/A_couple_of_tricks_with_the_secure_shell
http://en.wikipedia.org/wiki/List_of_file_systems#Distributed_fault-tolerant_file_systems
samba (AFAIK >v3.2) can do encrypted network transport if configured correctly, so you wouldn't need a tunnel at all.
but, because SMB is designed for sharing with dos/windows one have to be careful in setting up name mangling and such things to not mess up filenames and permissions and thats why I didn't mentioned it in first place.
On my personal network storage (not used in production) I just used rsync and samba.
Never had any issues but not sure if this would be the direction you wanted to go (and bandwidth isn't a concern for me).
I wouldn't run samba, nfs, etc. over a WAN. They're very "chatty" protocols and were never designed to run over a WAN.
If you're just trying to keep a directory on two boxes in sync over a WAN, your best bet by far for something easy/fast is bittorrent sync. It's relatively secure, it does checksumming, and it is bidirectional.
http://askubuntu.com/questions/284683/how-to-run-bittorrent-sync
Its tunneled
tunnel it and use nfs or samba or sshfs, depends on what you want to do.
have a shared directory on a third server that is accessible from both dedis or just want to save files on each dedi?
It's not just the security (though that prevents eavesdroppers) - it's the way those protocols work:
http://windowsadminguru.blogspot.com/2008/07/wide-area-file-sharing-across-wan.html
Quoting from that:
Let's take a look at a closer look at the NFS protocol to show an example of this type of "chatty" behavior. For certain operations on a filesystem using NFS (such as an synchronization of a source code tree), almost 80% of the RPCs sent across the network can be access RPCs, while the actual read and write RPCs typically comprise only 8-10% of the RPCs. Thus, 80% of the work done by the protocol is simply spent trying to determine if the NFS client has the proper permissions to access a particular file on the NFS server, rather than actually moving data.
In a LAN environment, these RPCs do not impact performance significantly, but when combined with the high latency typical of WANs, these RPCs can be deadly to performance. Worse, remote clients often end up timing out and retransmitting the RPCs, compounding the inefficiency. Furthermore, because data movement RPCs make up such a small percentage of the communication, increasing network bandwidth will make no difference to the aggravated end user. Like NFS, CIFS and IPX/SPX suffer from issues of "chattiness" that negatively impact performance over the WAN.
That article is from 2008 (given the url) and SMB has had some improvments over the years.
I haven't looked into it more then scratching the surface but I sure hope that it less "chatty" today.
Still haven't read the linked article but changes were made in the 2-2.1 version of samba/cifs to improve network bandwidth usage.
If you are using Windows 2012 or any other OS that supports smb3.2 I would suggest to disable smb1 which is obselete.
Use SyncThing.