Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


How to set up your own distributed, redundant, and encrypted storage grid in a few easy steps - Page 2
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

How to set up your own distributed, redundant, and encrypted storage grid in a few easy steps

2»

Comments

  • joepie91joepie91 Member, Patron Provider

    @craigb said: @joepie91 also, isn't it the case that by default, nodes closest in latency terms get filled up faster on average?

    No. Nodes are, as far as I am aware, only chosen by latency when downloading. Uploading will happen with deterministic randomness - as it should, because if the storage servers were picked on basis of latency, it would create a single (geographical) point of failure.

  • @joepie91 that's good to know - thanks!

  • joepie91joepie91 Member, Patron Provider

    That being said, if you're planning on for example building a CDN with Tahoe-LAFS as backend, you'll probably want to make sure that you either have an expansion factor of at least 3, or heavy caching, so that it's likely that data can be retrieved entirely from the same geographical area as the request originates from :)

  • @joepie91 said: building a CDN with Tahoe-LAFS as backend

    I like this.

  • RaymiiRaymii Member
    edited November 2012

    I also like this. Mirrored it at https://raymii.org/cms/p_Tahoe_LAFS_set_up_your_own_distributed_redundant_and_encrypted_storage_grid

    btw, I once also had vps's that where part of the CCC storage grid...

  • Very good, thanks for posting this. Will read properly and have a play with it myself later. Just what ive been wanting to do with a few of my boxes.

  • @joepie91 said: I'm not sure who packages this, so I would be careful :)

    Since you're using pip to install tahoe-lafs, don't you have to trust the pip and whoever packaged tahoe-lafs? Either way, you have to trust someone... I'd much rather it be a debian package, which can be verified against the original source pretty easily.

    Also, why use pip to install twisted rather than using the python-twisted debian package?

    Overall though, great guide!

  • joepie91joepie91 Member, Patron Provider

    @NickM said: Since you're using pip to install tahoe-lafs, don't you have to trust the pip and whoever packaged tahoe-lafs? Either way, you have to trust someone...

    The PyPi package (which pip uses) is packaged by zooko and warner, who are actually Tahoe-LAFS developers :)

    @NickM said: Also, why use pip to install twisted rather than using the python-twisted debian package?

    Because this way the dependency management is up entirely to the same package manager you're using to install Tahoe-LAFS - meaning that if for example a Twisted update is required, this can theoretically be done by pip itself. Twisted should be automatically installed when installing Tahoe-LAFS, but for some reason that dependency isn't working quite right, so I've included it as a separate installation step.

  • @joepie91 said: Because this way the dependency management is up entirely to the same package manager you're using to install Tahoe-LAFS - meaning that if for example a Twisted update is required, this can theoretically be done by pip itself. Twisted should be automatically installed when installing Tahoe-LAFS, but for some reason that dependency isn't working quite right, so I've included it as a separate installation step.

    It's almost like a gem install :P

  • By the way @joepie91 your article is very popular:

    image

  • Can anyone here comment on the performance of Tahoe? I'm thinking of using either Tahoe or GlusterFS, but read/write speed is important, and I can't seem to find a lot of performance-related info on Tahoe. I like the p2p design of Tahoe, but it has to be fast too.

Sign In or Register to comment.