All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
VPS Needed for Large Data Transfer Job (~1.5TB/day for 2-3 months)
Looking for a VPS to complete a data transfer sync from one Google Drive account to another (approx. 60TB in total) using rclone. Specs don't really matter, but the VPS will use a consistent 150 Mbit/s (hard limit) for about 2-3 months until the job is complete.
VZ Type: Doesn't matter
Number of Cores: Doesn't matter
RAM: >=512MB
Disk Space: Doesn't matter
Disk Type: Doesn't matter
Bandwidth: Unmetered or ridiculously high, like 60TB
Port Speed: >=200 Mbps
DDoS Protection: Doesn't matter
Number of IPs: 1 IPv4
Location: Doesn't matter
Budget: Hoping to keep this <=$5/mo, but flexible
Billing period: Monthly
Please post offers and recommendations. Please don't post if you aren't sure if network "fair use" policies will permit this amount of usage. Data isn't being stored locally, so disk size doesn't matter. The data is being encrypted upon upload, but even one CPU on an atom machine would be plenty. Port speed and bandwidth are really the only things that matter for this. Would like to have the VM spun up and rolling by EOD tomorrow (23 Aug). Thanks!
Comments
If my /r/datahoarder memory serves me right, I think people generally used scaleway for this type of transfer. Or Google Cloud because of the free internal bandwidth.
Scaleway might do the job nicely if they don't send passive aggressive emails to cut back on network usage (I know online.net "unlimited" servers are subject to a fair use policy). The baremetal C1 ARM box would probably do the job at €2.99/mo. I don't particuarly like online.net after the whole price raise a couple months after the promo they did a while back, so this will be my fallback should nothing else work out. Thanks for the recommendation!
Google Cloud scares me a bit due to their complex pricing and crazy bandwidth overage bills. Indeed, all traffic should be internal to Google, but for peace of mind I'd rather stay away and fork out the $5/mo instead.
Thanks again!
I have used Tilaa.com for several years and they are solid. They are what comes to my mind when unlimited traffic is mentioned I'll paste their cheapest specs below.
Server € 6.00 / mo
Memory 1 GB
Storage 20 GB HDD
Swap 1 GB
CPU cores 1 × 2.4Ghz @ 80%, included
Multiple snapshots yes / first is free
Network traffic unlimited / free
Edit: It's KVM.
Be sure to note that it is NOT unlimited traffic but 'fair-share' and they spell this out very specifically in their AUP / TOS. Their definition of 'fair-share' is 5TB, if you exceed that and become abusive they will write to you and request you stop the traffic, upgrade, or agree to be limited. I have only ever got around the limit once and been notified, generally if your realistic and not abusive they won't bother you and their routing is quite good.
That said, I wouldn't use the above outlined server for what you are wanting as it sounds like you would quickly exceed that 'fair-share' limit.
my 2 cents.
Cheers!
Maybe check with @exception0x876 which of Wishosting's unmetered 250/500M nodes have lowest average utilization, and order that plan
As I understand you won't be thrashing disks; just using RAM, 150mbit sustained (75+75?) and a fraction of the available cpu crypto capacity (decrypt+encrypt combined).
My storage VM in CA always seems to have ~500mbit available .
Two OVH VPS at 100 mbit each. It's crazy if there's no way to do that transfer completely inside google though. Or that there's scary billing for Google Cloud but not Google Drive.
I can do it in CZ for 4.5€/month (~$5). I have Google on IX so traffic to it is basically free (not to mention it's a direct ~0.7ms route), for your purpose I can hard limit you to 350Mbps and feel free to max it out 24/7.
@MasonR there's an unmetered 200Mbps offer here:
https://www.lowendtalk.com/discussion/159879/kvm-vps-just-1-4-mo-200-mbit-with-unlimited-bandwidth
Sorry about doing the only thing you didn't want to be done
I'm hitting around 1.4TB/day (750GB IN+OUT) with BuyVM $2/m slice and haven't had any issues
@MasonR If it's mostly .iso files why not do the native file copying implemented in Google Drive? On your main gdrive account share the folder(s) with all the files with your backup gdrive account.
Once it's shared, login with your backup gdrive account and add the shared folder to quick access. Now, ctrl+a select all files/isos in each folder, right click and select "make copy". Now it will make a copy of all the selected files in your local (backup) gdrive account. Once done, you can disable the shared folder again on your main account. Basically this https://zapier.com/learn/how-to/copy-google-drive-folders/ just with a shared folder.
I know this takes some manual work, but it would keep bandwidth internal. Just an idea. Perhaps I'm a fool as I've never done it to that extent but moving the files over a couple months via vps seems also quite the hassle.
Please no, its fucking clickbait, thats all it is.
Good to know.
Just pick the french, they are fine if you use 200Mbit for porn transfer all the day.
I can vouch for this.
If you really don't want to pay offline.net, you may consider get a slice from buyvm. I recommend to check with @Francisco before, but a 2GB should be ok with 200 Mbps all days.
I would suggest BuyVM as well. As far as I know the slices have unmetered bandwidth.
I would've suggested BuyVM as well, but afaik you're supposed to keep your 24/7 utilization around 100Mbps per 4GB of ram - and a 4GB slice is 3x his budget. Would need to ask Fran for an exception to do it on a smaller plan I think.
Google Cloud.
You may even already have free credits. Block all external traffic via rules and copy away; should be super-fast too.
Otherwise OVH and subsidiaries, they really don't seem to care if you hit your limit 24/7. Just don't expect burst speed to be available afterwards (which doesn't seem to matter anyway)
At one point you could even server-side copy large amounts of data - but that now seems to have a limit, even lower than the 750GB/day one. That's still per "user", however.
With Google Cloud you'll be able to do 60TB in much less time with certain configurations, if time is important.
I've synced around 20 TB using rclone on the free tier with no charges. It will spike to 50-80MB/s for the first minute or so before sustaining around 20MB/s. This is on us-east.
@willie - considered that, but would rather keep things simple on a single machine. Fire and forget is the goal.
@bugrakoc - heh, no worries
@sanvit + @beagle - as @Harambe mentioned, I believe there's some implied limits (not sure what they are, though). Maybe Fran can shed some light. I do already have a $2/mo slice for the Blesta goodness, though.
@Ksygo - time isn't important, so I'd rather not do the team drive/service account trickery. Or anything manual for that matter. A simple sync is perfect as I don't really care about how long it takes and I can limit it just under 750GB/day so I don't get the account temp banned.
Would rather do it outside of google cloud for peace of mind. If I need to hire a lawyer and two financial advisors to decipher their pricing, I'd rather just stay away even if it'll end up being free.
@CyberneticTitan - it's gonna take the same amount of time whether I do it at full speed and cut it off or just do a slow sync due to the 750GB/day upload limit per account.
This sounds like 100mbps is enough. It's about 1TB/day. Oh hmm, don't know if it's full duplex: I guess I could try on one of my kimsufi dedis.
Do what @Kysgo recommended, I done the exact same thing. It transfers at 1G down and 1G up and bandwidth isn't counted within the Google universe so it's free of charge if you have free credits.
hetzner cloud instance 30TB month ? With the cap at 750GB/day, it shoul be ok.
It's transferring from one account to another, so essentially 750GB download + 750GB upload daily
Needs to be limited to not breach the daily cap. Speed isn't the issue as long as it allows 8.5MB/s down + 8.5MB/s up sustained.
Hmm as someone else says, Hetzner Cloud maybe? I think it's limited to 20TB/m outbound, but inbound doesn't count. Port speed is now 1gbit I think. Was 10gbit in the past.
@Francisco did say he's OK with pushing 150TB on some other thread (not sure of the exact pricing of the slice though). Maybe a little hint please fran?
We'd be glad to sponsor this if any LET provider wants to chime in and make an offer. @MasonR is a giver, not a taker. Please, someone get him operational within his deadline of tomorrow and we'll foot the bill. Premium guys welcome. No budget cap as long as it's reasonable.
His call on who he chooses.
@Clouvider @Francisco @intovps @HostDoc tagging you because of his short deadline. We'll take care of it if you can help him out. Sorry if I missed others.
@masonR - I can put you on a very quiet node if you want to pickup a 512MB or similar with a slab.
Francisco