Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Noob questions on storage
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Noob questions on storage

migasllmigasll Member
edited November 2016 in Help

Hello,
I'm new to managing my own infrastructure and I'd like some input from more experienced people.
I have a vps that has 2gb ram and 40gb ssd with:
GOGS git repository (~50mb ram)
NEXUS repository (~1gb ram... YES THAT MUCH)
Jenkins (~300mb ram)
These 3 are run from docker containers.

I sometimes have other docker containers that are basically jenkins workers or other small tests I'm doing.
Then I have some small things like openvpn, fail2ban and stuff like that.

Everything runs great, I have enough CPU and memory.
The problem is storage, of course. I'll rapidly run out of storage.

So my question is if it's stupid to run GOGS, NEXUS and JENKINS containers from my main VPS but then remote mount their storage folders on a 3€/month time4vps 1Tb vps.

And also, I'm very concerned about backups. My set up right now is like this:
I rsync my vps 3 important folders (nexus, jenkins and gogs) to 3 windows shared folders of my desktop, through cifs, through my openvpn. I have enough storage on my desktop so this works. Is this stupid too?

Thanks

Comments

  • alaningusalaningus Member
    edited November 2016

    Minimum 1TB at time4vps now I'm afraid

  • AnthonySmithAnthonySmith Member, Patron Provider
    edited November 2016

    I guess the most simple way forward for this is to use sshfs, as your main VPS is KVM so this is fine however I think time4vps is OpenVZ which will limit your options unless they are happy to enable fuse for you.

    If they will then yes you could create a directory and mount the time4vps storage on your kvm vps as slow storage.

    Your backup solution is not stupid, not sure why cifs comes in to play though?

    for windows you could just install delta copy which is a GUI based rsync client/server (you only need the client), and you can schedule it to pull your files from your server using rsync.

    I guess the thing to work out is how much extra storage do you actually need?

  • @AnthonySmith said:
    I guess the most simple way forward for this is to use sshfs, as your main VPS is KVM so this is fine however I think time4vps is OpenVZ which will limit your options unless they are happy to enable fuse for you.

    If they will then yes you could create a directory and mount the time4vps storage on your kvm vps as slow storage.

    Well, I guess you mixed something up there: You don't need FUSE support on the machine that you are going to mount. ;)
    Mounting my Time4VPS storage on KVM, working great so far.

    @migasll: If I were you, I'd make a few test runs to see how the applications behave with their working directories mounted over SSHFS.

    Thanked by 1AnthonySmith
  • AnthonySmithAnthonySmith Member, Patron Provider

    ah right, your not wrong @Bochi :)

  • @migasll said:
    So my question is if it's stupid to run GOGS, NEXUS and JENKINS containers from my main VPS but then remote mount their storage folders on a 1€/month time4vps 500gb vps.

    It's stupid to run containers at all in a resource-constrained environment. They exist to solve a different problem.

    I rsync my vps 3 important folders (nexus, jenkins and gogs) to 3 windows shared folders of my desktop, through cifs, through my openvpn. I have enough storage on my desktop so this works. Is this stupid too?

    Probably. It all depends on what alternatives you'd consider. Many shops are set up so that development isn't done on production servers, so the backup needs are orthogonal.

    Thanked by 1vimalware
  • remote mounting from Lithuania?

    Where's your VPS located?

    Take a look at netcup.de 's new KVM offers. 2 dedicated logical cores, 3GB ram, 240GB SAS RAID10 for around 50 euros yearly(if you're outside EU) :

    https://www.netcup.de/bestellen/produkt.php?produkt=1607

  • migasllmigasll Member
    edited November 2016

    @alaningus said:
    Minimum 1TB at time4vps now I'm afraid

    From their website, there's the 0.5Tb option if you pay monthly.

    @AnthonySmith said:
    Your backup solution is not stupid, not sure why cifs comes in to play though?

    I'm using cifs because it supports windows natively. It has samba 2. So all I need on windows side is a shared folder. So what I do is to mount the windows folder on the vps and then rsync the important folder to there. The only downside is that it doesn't keep permissions and linux properties of the files (which is actually a very big downside)

    @AnthonySmith said:
    for windows you could just install delta copy which is a GUI based rsync client/server (you only need the client), and you can schedule it to pull your files from your server using rsync.

    Thanks, I'll look into that!

    @AnthonySmith said:
    I guess the thing to work out is how much extra storage do you actually need?

    I'm not sure how much do I need, but I want my main vps running on SSD storage and I guess it would be too expensive to upgrade. With time4vps I'd only pay 1€ for 500gb, I think no one can compete with that and I'd probably never have to worry about storage for a while.

    @Bochi said:
    Mounting my Time4VPS storage on KVM, working great so far.
    @migasll: If I were you, I'd make a few test runs to see how the applications behave with their working directories mounted over SSHFS.

    Thanks a lot for your input!

    @impossiblystupid said:
    It's stupid to run containers at all in a resource-constrained environment. They exist to solve a different problem.

    Thanks, but I don't really agree with you. Docker containers add next to zero overhead (I've read some reports on this) and the benefits they add, in my case, are invaluable. Before using docker I had a million ansible scripts to deploy my servers (which in itself is already better than doing it from scratch) and it was a nightmare to backup all the important folders, configurations, etc. Now, with docker, all I need to do is to backup 1 folder for each container and, in a new server, I just need to run the images (which i don't even need to backup) and I'll have exactly the same service running. It's just too easy not to use it. Also, it adds another layer of protection, isolating the service.

    @impossiblystupid said:
    Probably. It all depends on what alternatives you'd consider. Many shops are set up so that development isn't done on production servers, so the backup needs are orthogonal.

    I think ideally I'd have my own NAS for backups, and it would be straightforward. But right now I can't do that, so I use what I have. I don't have a production and development server, just the one VPS, and a local Windows Desktop with a VMWare Centos7 in it.

  • @migasll said:
    From their website, there's the 0.5Tb option if you pay monthly.

    I think you might want to check that. As far as I can see they don't do 0.5TB (as stated several times on here by their rep) and if you pay monthly the smallest is 2TB

  • migasllmigasll Member
    edited November 2016

    @alaningus said:

    @migasll said:
    From their website, there's the 0.5Tb option if you pay monthly.

    I think you might want to check that. As far as I can see they don't do 0.5TB (as stated several times on here by their rep) and if you pay monthly the smallest is 2TB

    EDIT:
    Yes, you are right, I made a mistake.
    their 1Tb (minimum) is 3€/month paid annually.
    I already edited the 1st post, thanks for the heads up!

    Thanked by 1alaningus
  • migasllmigasll Member
    edited November 2016

    Sorry for this post, I don't know how to delete

  • Side Note:

    If you find yourself always running out of space, want to fiddle with things, and want to have redundancy (which is not a backup...), it might be worth it to look at some distributed storage systems such as GlusterFS, Ceph, or XtreemFS. Assuming you have a lot of large files, if you do replication with sharding (Sorry, only familiar with GlusterFS), that should enable you to have a nice amount of storage while keeping redundancy.

  • time4vpstime4vps Member, Host Rep

    Hi, as I see there is some questions regarding our servers (there are some correct answers, as well as some quite not correct), so I will clarify.

    • We are offering three storage server packages: 1 TB (only for annual and biennial billing cycles), 2 TB and 4 TB. Unfortunately there is no 0.5 TB package anymore for some time now.

    • We do not support FUSE on our servers, we recommend to use NFS instead.

    • However, it is possible to mount our server to another server which has FUSE enabled. In other words, FUSE needed only for a host server, but not for one you mount.

    • We are using OpenVZ virtualization for all our servers right now. However we should start offering KVM based servers as well, it should be in the couple of weeks.

Sign In or Register to comment.