New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Storage VPS/server?
This discussion has been closed.
Comments
What about creating your own using those online.net servers. 8TB (7.5?) usable in raid5 for 50/month
We dont need that much, 1TB is more than enough for now, our budget doesnt allow us to go for that route either. We are using around 200GB, though I predict we will be using 1TB by the end of next year.
Maybe https://www.hetzner.de/storage-box is the most reliable if is only for backup data.
SFTP support, that's nice. I haven't seen many others that have it available unless I'm blind.
Thank you, but we use minio, so definitely need a VPS/Dedicated
If you need that much iops much of the time, you need a dedi or a high resource storage server. Both will be out of your price range for a tb of storage, unless you can snag a Dedibox SC or similar.
I've been happy with ikoula's €8.99/month "green fish" dedi promo: 1 tb hdd 4 gb ram atom 510 (dual core 1.6 ghz), 100 Mbps somewhere in France - seems pretty solid for the 2 months I've been using it. I installed minio on it but haven't exercised it much or scrutinized IOPS. Seems like it might be suitable for your use and (almost) within your stated budget.
I believed old offer from @AshleyUK still available
Order link: https://billing.zxhost.co.uk/cart.php?a=add&pid=63
KVM, 1.1TB Storage, unlimited incoming and 2TB outgoing. 96$/3years.
If he has a problem with a 200 IOPS limit then a dedicated server with a single drive isn't going to help since most consumer HDD drives won't provide much more than 100 IOPS.
@AshleyUK Sound like a job for you. https://zxhost.co.uk
Thank you, but we use minio, so definitely need a VPS/Dedicated> @willie said:
Not that much. It's just that Time4VPS keeps throwing lots of errors, we're talking about 73GB that we're trying to sync now, and the iops aren't enough.
I'll check it right now, thank you! :-)
As much as I love Ashley's offer, we'd rather try for a month then see how it goes and hopefully renew it for a year.
This is not the case here. I assure you their limits aren't enough if you're running minio... We're trying to sync 73GB, and 10 transfers per second, I can't just limit it o 1 transfer per second. I've tried this using smaller VPS'es and were able to, no iowait, nothing. It's just too limited nowadays in terms of IO performance.
Will check!
We were very happy with Amazon Cloud Drive until they banned rclone and related...
Hubic suspended our account even though we had paid for it on time, and we're yet waiting to receive their reply to the email we sent them like 4/5 months ago.
It might be worthwhile to drop Ashley a pre-sales ticket or a PM; he might be willing to give you a one-month trial of some sort. I've always found them quite easy to work with.
Yeah well, we're getting like, 40Mb/s dd test (not accurate at all) but we aren't doing any intensive uploads as well, and even kept it to 10 transfers.
I'll PM Ashley.
Keep in mind the ZHost special is Ceph network storage; they did a good job implementing it, but it's not the same as local disk.
I haven't had any problems with using T4V for overnight backups, though. When importing backups I needed to rsync about 200GB; I don't recall how long it took, but within a few hours I think.
I did a 1TB transfer from a Hetzner dedi to ZXHost Ceph storage (filling up the VPS) and got the full 1Gb/s pretty much the whole time. I was impressed.
I've got an obsolete KS3c. Probably can't transfer but I could give you that without markup. Downside is it's only 100Mbit.
Thank you for the information, just received a reply from Ashley
Thank you very much, I appreciate. Its not transferable and only 100mbit. We got more upload than that. Well, will have!
But still, thank you for the gentle offer
Would it be possible for you to run this small test on your ZXHost VPS?
Could you elaborate on those errors a bit more?
If a consumer drive on a dedi is still too slow (ram caching helps a lot) then your budget is unrealistic. Reconsider your software approach or be ready to spend a lot more.
"Storage server" unfortunately seems to mean two things here on LET: 1) a place where you store stuff for occasional access and updates, analogous to a storage locker or shed where you put infrequently used household stuff; 2) a high performance server that happens to have a lot of disk space as well. Too many people buy #1 and expect to get #2. #2 costs a lot more.
If you're going to use this box heavily and you're doing something serious with it, consider getting a real budget. A Hetzner EX41-SSD has a fast cpu, 32gb ram, and 2x 500GB SSD drives with all the iops you'll ever want, all for 39 euro/month (plus 79euro setup) ex vat, which seems like a terrific deal to me. There's lots of other ssd options too. But I'm surprised if you can't get by with 1 or 2 normal hard disks and adequate ram caching.
Already have by ticket, and it seems it's not negotiable. So, no furher discussion needed.
IO errors, while verifying the file's hash. I've limited it to 5 transfers, slower, but seems to be working.
Ooops, maybe I did not make myself clear. They didn't suspend our account for the IO usage, because our IO usage is not that intensive. We got 73GB used for now, planning to have 1TB in the next year, since it's all Design work and we got all files here, EPS, etc.
Hubic suspended our account for non payment, though the payment was done. We sent them an email months ago and they haven't bothered to get back to us, they do still send us invoices, not sure why, anyway.
I'm an experienced DevOp and know what I am talking about. I appreciate your input but I am very aware of the issue.
Doing hashing verification 10 files per second, gets this VPS throwing IO errors all the time. That's the issue here. It's not that IO intensive.
The HDD that has the backups (in our office), its a standard desktop HDD, 3.5", and we used to do 30/40 threads / hash file verification per second just fine, similar results would happen with rsync if you don't limit it to a lower number of hash verifications, now this isn't working properly with Time4VPS and the IO Usage isn't that high, at all. It's quite low actually. We experience iowait upon transferring a few files, only a few MB's per second.
We have, as of now, 10 VPS with Time4VPS. The latest VPS purchases has their new IO limits and it's definitely performing worse than the legacy plans, which we've kindly asked them to keep the old limits. They did mention there is no space for negotiation unless I pay 3x the value I'm paying right now, which wouldn't be worth at all. I mean, for 66 EUR (with all discounts applied) for 6 month, we can just go for a kimsufi. After all, this has 100mbps as well instead of the previous 400mbps.
I'm not even into fighting this, not interested to. We've used the VPS for about a month and a half, and it's pre-paid for 6 months, we just want to switch to a decent provider whom actually bothers looking into the issue and improve their customer service.
We're talking about 73GB, avg file size has, maybe 10MB, we're hosting some stuff here, but it is NOT that IO intensive, at all.
If this runs in our cheap 3TB HDD just fine, syncing to Amazon or even Hubic with 30/40 threads / hash verifications, then it should run well in Time4VPS as well, given their advertised 200 IOPS.
Furthermore, here's the classic dd test, just to have an idea. No cache enabled on our HDD, won't get FIO (which is obviously more accurate, not even comparable) as I don't really want to spend much more time with this:
Our HDD:
root@srv:/media/disk1# dd if=/dev/zero of=test bs=64k count=16k conv=fdatasync
16384+0 registos dentro
16384+0 registos fora
1073741824 bytes (1,1 GB) copiados, 7,97636 s, 135 MB/s
Time4VPS, not being used at all:
root@int-cloud:~# dd if=/dev/zero of=test bs=64k count=16k conv=fdatasync
16384+0 records in
16384+0 records out
1073741824 bytes (1.1 GB) copied, 18.5616 s, 57.8 MB/s
Our HDD IOPing:
--- . (ext4 /dev/sdb1) ioping statistics ---
10 requests completed in 9.0 s, 283 iops, 1.1 MiB/s
min/avg/max/mdev = 115 us / 3.5 ms / 16.0 ms / 5.5 ms
Time4VPS, not being used at all, but simfs effect:
--- . (simfs /dev/simfs) ioping statistics ---
10 requests completed in 9.07 s, 11.1 k iops, 43.5 MiB/s
min/avg/max/mdev = 75 us / 89 us / 105 us / 10 us
11.1k iops, of course, absolutely...
I hope this clears what I meant here, and no further discussion needed. This topic was open to find an alternative solution (which seems to have resulted as I've got useful replies here with suggestions for providers, as well as a few PM's), I'll rather drop Time4VPS and seek for service somewhere else. Pretty sure it won't be an issue.
root@de2:~# dd if=/dev/sdb of=/dev/null bs=1M count=20480 20480+0 records in 20480+0 records out 21474836480 bytes (21 GB, 20 GiB) copied, 8952.53 s, 2.4 MB/sNot that small, 20GB....
No you do not confirm... ;-P. We're seeking a special @ online.net's, or we'll go with Ashley or someone else. But we definitely do not need a Dedicated Server for 73GB of EPS files.
I know. Its that read speed (2.4 MB/s) that bothers me. Thanks @advarisk for those test results. I have been facing some strange read issues for my particular use case so just wanted to see if the output of this test is similar for others too.
Wait earlier you said 1TB? If it's about 73GB try a Scaleway VC1S or whatever they call it now. 3 euro/m for 2 Atom C2750 cores, 2GB of ram, and 50GB of SSD, plus 1 euro/month per additional 50GB chunk. It's network SSD but should still have more iops than any HDD.
Again I don't think your expectations are realistic for a cheap storage VPS. You're comparing it to your local HDD which is effectively a dedi. But other people will be using that VPS hardware at the same time as you. You say "Time4vps not being used at all" but it's shared hardware, so of course it's always being used by someone.
We've been rapidly expanding and I don't want to be changing servers all the time, so yeah I'll go for 1TB.
Trying to grab an online.net's special, if not, I already who I should signup for. :P
You're definitely not getting my point. Run minio.io there, then upload with rclone (10 transfers), you'll see that it's not even enough for the checksums.
BTW thread can be closed since I've already got a few options to choose from - thank you for your help!
Can't add/change the storage on a VC1S
You mean Scaleway? Hmm that's interesting. I've used them for a while but haven't done anything that i/o intensive. I'll be interested to hear if you have better luck with other storage VPS.