Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Subscribe to our newsletter

Advertise on LowEndTalk.com

Latest LowEndBox Offers

    VPS Needed for Large Data Transfer Job (~1.5TB/day for 2-3 months)
    New on LowEndTalk? Please read our 'Community Rules' by clicking on it in the right menu!

    VPS Needed for Large Data Transfer Job (~1.5TB/day for 2-3 months)

    Looking for a VPS to complete a data transfer sync from one Google Drive account to another (approx. 60TB in total) using rclone. Specs don't really matter, but the VPS will use a consistent 150 Mbit/s (hard limit) for about 2-3 months until the job is complete.

    VZ Type: Doesn't matter
    Number of Cores: Doesn't matter
    RAM: >=512MB
    Disk Space: Doesn't matter
    Disk Type: Doesn't matter
    Bandwidth: Unmetered or ridiculously high, like 60TB
    Port Speed: >=200 Mbps
    DDoS Protection: Doesn't matter
    Number of IPs: 1 IPv4
    Location: Doesn't matter
    Budget: Hoping to keep this <=$5/mo, but flexible
    Billing period: Monthly

    Please post offers and recommendations. Please don't post if you aren't sure if network "fair use" policies will permit this amount of usage. Data isn't being stored locally, so disk size doesn't matter. The data is being encrypted upon upload, but even one CPU on an atom machine would be plenty. Port speed and bandwidth are really the only things that matter for this. Would like to have the VM spun up and rolling by EOD tomorrow (23 Aug). Thanks!

    LowEndStorage | FreeMach - Free KVM Servers (LET Thread)
    Yet Another Bench Script (YABS) - Linux Server Benchmarking Script (dd, iperf, & Geekbench)

    «1

    Comments

    • If my /r/datahoarder memory serves me right, I think people generally used scaleway for this type of transfer. Or Google Cloud because of the free internal bandwidth.

      Thanked by 4MasonR Ksygo dosai 479555

      🐴 Recommended: $20/yr 512MB KVM - Unmetered bandwidth. $5/TB Block Storage - from BuyVM (aff)

    • @Harambe said:
      If my /r/datahoarder memory serves me right, I think people generally used scaleway for this type of transfer. Or Google Cloud because of the free internal bandwidth.

      Scaleway might do the job nicely if they don't send passive aggressive emails to cut back on network usage (I know online.net "unlimited" servers are subject to a fair use policy). The baremetal C1 ARM box would probably do the job at €2.99/mo. I don't particuarly like online.net after the whole price raise a couple months after the promo they did a while back, so this will be my fallback should nothing else work out. Thanks for the recommendation!

      Google Cloud scares me a bit due to their complex pricing and crazy bandwidth overage bills. Indeed, all traffic should be internal to Google, but for peace of mind I'd rather stay away and fork out the $5/mo instead.

      Thanks again! :)

      LowEndStorage | FreeMach - Free KVM Servers (LET Thread)
      Yet Another Bench Script (YABS) - Linux Server Benchmarking Script (dd, iperf, & Geekbench)

    • bugrakocbugrakoc Member
      edited August 2019

      I have used Tilaa.com for several years and they are solid. They are what comes to my mind when unlimited traffic is mentioned :smile: I'll paste their cheapest specs below.

      Server € 6.00 / mo
      Memory 1 GB
      Storage 20 GB HDD
      Swap 1 GB
      CPU cores 1 × 2.4Ghz @ 80%, included
      Multiple snapshots yes / first is free
      Network traffic unlimited / free

      Edit: It's KVM.

      Thanked by 1MasonR
    • TheLinuxBugTheLinuxBug Member
      edited August 2019

      bugrakoc said: I have used Tilaa.com for several years and they are solid. They are what comes to my mind when unlimited traffic is mentioned I'll paste their cheapest specs below.

      Be sure to note that it is NOT unlimited traffic but 'fair-share' and they spell this out very specifically in their AUP / TOS. Their definition of 'fair-share' is 5TB, if you exceed that and become abusive they will write to you and request you stop the traffic, upgrade, or agree to be limited. I have only ever got around the limit once and been notified, generally if your realistic and not abusive they won't bother you and their routing is quite good.

      That said, I wouldn't use the above outlined server for what you are wanting as it sounds like you would quickly exceed that 'fair-share' limit.

      my 2 cents.

      Cheers!

      Thanked by 1MasonR

      Have an Allwinner H3 device? Android? Check out H3Droid! | Lichee Pi Zero - The 6$ SBC | #SYSarm - Get It! | Atomic Pi - $35 x86 SBC
      21+ Years IT Experience in Linux/Windows Hosting, Administration and Development Services

    • Maybe check with @exception0x876 which of Wishosting's unmetered 250/500M nodes have lowest average utilization, and order that plan

      As I understand you won't be thrashing disks; just using RAM, 150mbit sustained (75+75?) and a fraction of the available cpu crypto capacity (decrypt+encrypt combined).

      My storage VM in CA always seems to have ~500mbit available .

      Thanked by 1MasonR

      Tired of LET scams?
      A Moderated forum : https://talk.lowendspirit.com/

    • williewillie Member
      edited August 2019

      Two OVH VPS at 100 mbit each. It's crazy if there's no way to do that transfer completely inside google though. Or that there's scary billing for Google Cloud but not Google Drive.

      Thanked by 1MasonR

      #lexit spread the word.

    • FHRFHR Member, Provider

      I can do it in CZ for 4.5€/month (~$5). I have Google on IX so traffic to it is basically free (not to mention it's a direct ~0.7ms route), for your purpose I can hard limit you to 350Mbps and feel free to max it out 24/7.

      SkylonHost - affordable hourly-billed KVM VPS in Prague, CZ!
      Featuring own high performance network AS202297 | RIPE NCC member | Contact us for IPs/ASNs

    • ITLabsITLabs Member
      edited August 2019
      Thanked by 1MasonR

      #lexit | FatPal - Official LET payment gateway

    • MasonR said: Please don't post if you aren't sure if network "fair use" policies will permit this amount of usage.

      Sorry about doing the only thing you didn't want to be done :tongue:

    • sanvitsanvit Member
      edited August 2019

      I'm hitting around 1.4TB/day (750GB IN+OUT) with BuyVM $2/m slice and haven't had any issues

      Thanked by 1MasonR
    • YmpkerYmpker Member
      edited August 2019

      @MasonR If it's mostly .iso files why not do the native file copying implemented in Google Drive? On your main gdrive account share the folder(s) with all the files with your backup gdrive account.
      Once it's shared, login with your backup gdrive account and add the shared folder to quick access. Now, ctrl+a select all files/isos in each folder, right click and select "make copy". Now it will make a copy of all the selected files in your local (backup) gdrive account. Once done, you can disable the shared folder again on your main account. Basically this https://zapier.com/learn/how-to/copy-google-drive-folders/ just with a shared folder.

      I know this takes some manual work, but it would keep bandwidth internal. Just an idea. Perhaps I'm a fool as I've never done it to that extent but moving the files over a couple months via vps seems also quite the hassle.

      Thanked by 1MasonR
    • Please no, its fucking clickbait, thats all it is.

    • @Neoon said:

      Please no, its fucking clickbait, thats all it is.

      Good to know.

      #lexit | FatPal - Official LET payment gateway

    • Just pick the french, they are fine if you use 200Mbit for porn transfer all the day.

    • NekkiNekki Moderator

      @Neoon said:
      Just pick the french, they are fine if you use 200Mbit for porn transfer all the day.

      I can vouch for this.

      Thanked by 2bugrakoc MasonR

      Here lies Nekki. He loved massive amounts of storage, K-Pop and calling people cunts.

    • If you really don't want to pay offline.net, you may consider get a slice from buyvm. I recommend to check with @Francisco before, but a 2GB should be ok with 200 Mbps all days.

      Thanked by 1MasonR
    • I would suggest BuyVM as well. As far as I know the slices have unmetered bandwidth.

      Thanked by 1MasonR
    • I would've suggested BuyVM as well, but afaik you're supposed to keep your 24/7 utilization around 100Mbps per 4GB of ram - and a 4GB slice is 3x his budget. Would need to ask Fran for an exception to do it on a smaller plan I think.

      Thanked by 2MasonR Ympker

      🐴 Recommended: $20/yr 512MB KVM - Unmetered bandwidth. $5/TB Block Storage - from BuyVM (aff)

    • Google Cloud.
      You may even already have free credits. Block all external traffic via rules and copy away; should be super-fast too.

      Otherwise OVH and subsidiaries, they really don't seem to care if you hit your limit 24/7. Just don't expect burst speed to be available afterwards (which doesn't seem to matter anyway)

      At one point you could even server-side copy large amounts of data - but that now seems to have a limit, even lower than the 750GB/day one. That's still per "user", however.

      With Google Cloud you'll be able to do 60TB in much less time with certain configurations, if time is important.

      Thanked by 1MasonR
    • I've synced around 20 TB using rclone on the free tier with no charges. It will spike to 50-80MB/s for the first minute or so before sustaining around 20MB/s. This is on us-east.

      Thanked by 3MasonR SirFoxy Ouji
    • @willie - considered that, but would rather keep things simple on a single machine. Fire and forget is the goal.

      @bugrakoc - heh, no worries :)

      @sanvit + @beagle - as @Harambe mentioned, I believe there's some implied limits (not sure what they are, though). Maybe Fran can shed some light. I do already have a $2/mo slice for the Blesta goodness, though.

      @Ksygo - time isn't important, so I'd rather not do the team drive/service account trickery. Or anything manual for that matter. A simple sync is perfect as I don't really care about how long it takes and I can limit it just under 750GB/day so I don't get the account temp banned.

      Would rather do it outside of google cloud for peace of mind. If I need to hire a lawyer and two financial advisors to decipher their pricing, I'd rather just stay away even if it'll end up being free.

      @CyberneticTitan - it's gonna take the same amount of time whether I do it at full speed and cut it off or just do a slow sync due to the 750GB/day upload limit per account.

      Thanked by 2beagle Ksygo

      LowEndStorage | FreeMach - Free KVM Servers (LET Thread)
      Yet Another Bench Script (YABS) - Linux Server Benchmarking Script (dd, iperf, & Geekbench)

    • williewillie Member
      edited August 2019

      MasonR said: limit it just under 750GB/day so I don't get the account temp banned.

      This sounds like 100mbps is enough. It's about 1TB/day. Oh hmm, don't know if it's full duplex: I guess I could try on one of my kimsufi dedis.

      #lexit spread the word.

    • Do what @Kysgo recommended, I done the exact same thing. It transfers at 1G down and 1G up and bandwidth isn't counted within the Google universe so it's free of charge if you have free credits.

    • hetzner cloud instance 30TB month ? With the cap at 750GB/day, it shoul be ok.

      Thanked by 2MasonR Hetzner_OL
    • MasonRMasonR Member
      edited August 2019

      @willie said:

      MasonR said: limit it just under 750GB/day so I don't get the account temp banned.

      This sounds like 100mbps is enough. It's about 1TB/day.

      It's transferring from one account to another, so essentially 750GB download + 750GB upload daily

      @tester4 said:
      Do what @Kysgo recommended, I done the exact same thing. It transfers at 1G down and 1G up and bandwidth isn't counted within the Google universe so it's free of charge if you have free credits.

      Needs to be limited to not breach the daily cap. Speed isn't the issue as long as it allows 8.5MB/s down + 8.5MB/s up sustained.

      LowEndStorage | FreeMach - Free KVM Servers (LET Thread)
      Yet Another Bench Script (YABS) - Linux Server Benchmarking Script (dd, iperf, & Geekbench)

    • williewillie Member
      edited August 2019

      Hmm as someone else says, Hetzner Cloud maybe? I think it's limited to 20TB/m outbound, but inbound doesn't count. Port speed is now 1gbit I think. Was 10gbit in the past.

      Thanked by 2MasonR Hetzner_OL

      #lexit spread the word.

    • @MasonR said:
      @sanvit + @beagle - as @Harambe mentioned, I believe there's some implied limits (not sure what they are, though). Maybe Fran can shed some light. I do already have a $2/mo slice for the Blesta goodness, though.

      @Francisco did say he's OK with pushing 150TB on some other thread (not sure of the exact pricing of the slice though). Maybe a little hint please fran?

    • We'd be glad to sponsor this if any LET provider wants to chime in and make an offer. @MasonR is a giver, not a taker. Please, someone get him operational within his deadline of tomorrow and we'll foot the bill. Premium guys welcome. No budget cap as long as it's reasonable.

      His call on who he chooses.

    • @Clouvider @Francisco @intovps @HostDoc tagging you because of his short deadline. We'll take care of it if you can help him out. Sorry if I missed others.

      Thanked by 3sanvit MikePT intovps
    • FranciscoFrancisco Top Provider

      @sanvit said:

      @MasonR said:
      @sanvit + @beagle - as @Harambe mentioned, I believe there's some implied limits (not sure what they are, though). Maybe Fran can shed some light. I do already have a $2/mo slice for the Blesta goodness, though.

      @Francisco did say he's OK with pushing 150TB on some other thread (not sure of the exact pricing of the slice though). Maybe a little hint please fran?

      @masonR - I can put you on a very quiet node if you want to pickup a 512MB or similar with a slab.

      Francisco

      BuyVM - Free DirectAdmin, Softaculous, & Blesta! / Anycast Support! / Windows 2008, 2012, & 2016! / Unmetered Bandwidth!
      BuyShared - Shared & Reseller Hosting / cPanel + Softaculous + CloudLinux / Pure SSD! / Free Dedicated IP Address
    • MikeAMikeA Member, Provider
      edited August 2019

      @DA_Mark said:
      @Clouvider @Francisco @intovps @HostDoc tagging you because of his short deadline. We'll take care of it if you can help him out. Sorry if I missed others.

      I'll gladly take a free DA license instead :wink:

      Edit - I'm not serious, don't worry, DirectAdmin is cheap and I respect dev costs. He has an offer from me and many others, I'm sure at the very least the Google Cloud thing would work for him.

      Thanked by 2DA_Mark MasonR
    • Thanks all! And thank you for the kind gesture, @DA_Mark! Going to review everything in the morning when I wake up and set up everything from there :)

      Thanked by 1Ympker

      LowEndStorage | FreeMach - Free KVM Servers (LET Thread)
      Yet Another Bench Script (YABS) - Linux Server Benchmarking Script (dd, iperf, & Geekbench)

    • if you can live/work with a natted VM I can set you up for free as I have quite some ressources spare on a servdiscount dedi (aka myloc, unmetered).
      port forwardings if needed shouldn't be a problem...
      can also set that speed limit for network on the node for you, so you don't need to worry.

      other than that I totally agree, hetzner cloud most likely is the cheapest option.

      just let me know ;-)

      most recommended Provider: First-Root KVM Power-Edition /w SSD
      UltraVPS.eu KVM in US/UK/NL/DE: 15% off first 6 month | Netcup VPS/rootDS - 5€ off: 36nc15279180197 (ref)

    • Well, we had MyLoc as a pure backup machine, since after a bit, the backups took way to long. Turns out, they throttled the traffic since it goes via Frankfurt.

      After a few days long ticket, they took care and removed the throttle.
      Even if its advertised as 1000Mbit Unlimited, which is again pure clickbait, like the the russian guy selling his VPS for 1.4$.

      But maybe they stopped doing it, no idea.
      I mean we had maybe 50GB per day or so, not much.

      Thanked by 2Falzo MasonR
    • FalzoFalzo Member
      edited August 2019

      @Neoon said:
      Well, we had MyLoc as a pure backup machine, since after a bit, the backups took way to long. Turns out, they throttled the traffic since it goes via Frankfurt.

      After a few days long ticket, they took care and removed the throttle.
      Even if its advertised as 1000Mbit Unlimited, which is again pure clickbait, like the the russian guy selling his VPS for 1.4$.

      But maybe they stopped doing it, no idea.
      I mean we had maybe 50GB per day or so, not much.

      thanks for the headsup!

      servdiscount has quite a big banner directly on the main page promising 300mbit guarantued and no throttling. though, I never get close to use this for more than a few hours every other month with my normal traffic... I also use that box only for storage, the rest is idling along.

      so who knows, myloc hasn't had the best reputation for years, maybe it really is just marketing and they start throttling anyway and blame whatever instead.
      on the other hand, if the usage is already going to be restricted to ~100ish mbit full duplex - how low do they want to go when having to justify it somehow ^^

      however I also agree that it might be more of a 'guarantue' after all, if you pick a provider with a clearly mentioned limit in terms of TB. 🤷‍♂️

      Thanked by 1Ympker

      most recommended Provider: First-Root KVM Power-Edition /w SSD
      UltraVPS.eu KVM in US/UK/NL/DE: 15% off first 6 month | Netcup VPS/rootDS - 5€ off: 36nc15279180197 (ref)

    • NeoonNeoon Member
      edited August 2019

      Well, its about 2 years ago, not that I remember that they had the 300Mbit thingy before. However, you can do it 2 ways, either you route "abusers" crappier like Online announced, cheap machines will be routed the cheapest way or you throttle them.

      No idea how servDiscount handles it now, I suppose they went away from the throttling idea.

      Thanked by 1Falzo
    • MikePTMikePT Member, Provider

      @DA_Mark said:
      We'd be glad to sponsor this if any LET provider wants to chime in and make an offer. @MasonR is a giver, not a taker. Please, someone get him operational within his deadline of tomorrow and we'll foot the bill. Premium guys welcome. No budget cap as long as it's reasonable.

      His call on who he chooses.

      ❤️❤️❤️

    • bacloudbacloud Member, Provider

      60TB = around 200Mbps monthly usage. 1Gbps = 300TB+. Budget - $5. Doe's anyone now where I can buy unlimited dedicated 1Gbps for $25/month, 10Gbps for $250 or 100Gbps for just $2500?

      Selling VPS to OP means losing money :)

      Thanked by 1MasonR

      High quality VPS just from $2.80 EU/USA .

      Need Custom server? Please contact us [email protected] or Skype Andrius.Bacoud Payment methods: BTC + Altcoins, Paypal, CC, Alipay, Webmoney and more

    • ehabehab Member

      @bacloud said:

      off course

      Thanked by 2simonindia MasonR
      • do not prepay > 1 year and check for reviews/support
      • only use monthly from a provider operating < 1 year 🍆
    • @bacloud said:
      60TB = around 200Mbps monthly usage. 1Gbps = 300TB+. Budget - $5. Doe's anyone now where I can buy unlimited dedicated 1Gbps for $25/month, 10Gbps for $250 or 100Gbps for just $2500?

      Selling VPS to OP means losing money :)

      Yes, that's why I chimed in asked for LET hosts to bring their offers (no budget limits) and we'll sponsor it.

      If all else fails we'll still take care of @MasonR but I'd rather support a LET provider

      Thanked by 2Ouji MasonR
    • bacloud said: 60TB = around 200Mbps monthly usage. 1Gbps = 300TB+. Budget - $5.

      The 60TB of google drive space for $5/m or whatever is an even more entertaining aspect of this. That's actual hard drives though I'm happy to see Google resources going to a useful purpose like LET pron collections instead of surveillance ;). The bandwidth by comparison is no big deal, it's just some wire in a data center that's likely already connected to google through peering.

      Thanked by 2vimalware MasonR

      #lexit spread the word.

    • HostDocHostDoc Member, Provider, Top Provider

      Sorry I am late.
      @MasonR I can do this in Germany if it is not too late.

      Thanked by 2ehab MasonR
    • Please note that if you're moving 60TB of data, you're using 120TB of traffic, so that's 200mbit up AND 200mbit down.

      Three netcup vpsses would do it though.

      Thanked by 1MasonR

      I like my uptime down low and my servers all hacked. Can see me droppin' twenty-fours with a router in the rack.
      Ya like ya Switch-Ports hot and ya servers all hacked. If ya pings real high and ya networks pitch black.

    • @HostDoc said:
      Sorry I am late.
      @MasonR I can do this in Germany if it is not too late.

      Doesn't have to be within OP's price range. We'll sponsor this at any (fair) price. Most important is that he can for-sure get the speed he needs for data transfer. Just want some offers posted so he can choose the one he prefers. :)

      Thanked by 2MasonR HostDoc
    • Thanks all for the interest! I've gone with an offer from @MikeA (ExtraVM). I already have services with him, so I know I'm in good hands, and he had some spare resources that I could tap into for this. I appreciate all the responses and offers, you guys are awesome!

      @DA_Mark said:
      Doesn't have to be within OP's price range. We'll sponsor this at any (fair) price. Most important is that he can for-sure get the speed he needs for data transfer. Just want some offers posted so he can choose the one he prefers. :)

      I've already had a chance to test with the ExtraVM VPS and it hits the speeds without any issues, so everything is up and running. Seems like it is a good fit. The bill was small enough that I took care of it for this month. I really appreciate the offer to sponsor this, if you feel so inclined you can talk to him about taking care of the additional 1 or 2 months it'll take for this to complete :) Thanks again, Mark!

      LowEndStorage | FreeMach - Free KVM Servers (LET Thread)
      Yet Another Bench Script (YABS) - Linux Server Benchmarking Script (dd, iperf, & Geekbench)

    • @teamacc said:
      Please note that if you're moving 60TB of data, you're using 120TB of traffic, so that's 200mbit up AND 200mbit down.

      Three netcup vpsses would do it though.

      It's ~75mbit up and ~75mbit down (inc. tcp and google api overhead) so it doesn't breach the 750GB/day upload limit. Basically I can move about 23TB/month, so it'll take about 3 months at that rate no matter where it's done

      LowEndStorage | FreeMach - Free KVM Servers (LET Thread)
      Yet Another Bench Script (YABS) - Linux Server Benchmarking Script (dd, iperf, & Geekbench)

    • HostDocHostDoc Member, Provider, Top Provider
      edited August 2019

      @DA_Mark said:

      @HostDoc said:
      Sorry I am late.
      @MasonR I can do this in Germany if it is not too late.

      Doesn't have to be within OP's price range. We'll sponsor this at any (fair) price. Most important is that he can for-sure get the speed he needs for data transfer. Just want some offers posted so he can choose the one he prefers. :)

      VZ Type: KVM
      Number of Cores: 1 [Fair Share (25%)] @4.0Ghz
      RAM: = 512MB
      Disk Space: 5GB
      Disk Type: SSD
      Bandwidth: 120TB
      Port Speed: =200 Mbps (Throttled)
      DDoS Protection: Yes
      Number of IPs: 1 IPv4
      Location: Germany
      Budget: = £4.00/m |$4.91/mo
      Billing period: Monthly

      Our Germany traffic does not use much bandwidth meaning you are safe with the allocation.
      It will be throttled to 200mbps on a node that has a 1gbps port.

      @DA_Mark You are truely a good guy.

      EDIT2: I was typing while @MasonR responded. Oops.

      Thanked by 3MasonR Ouji ITLabs
    • intovpsintovps Member, Provider

      I'm glad you found a solution. I'm curious if Google will let you use 150 Mbit/s constantly.

      And btw, doesn't this work with share + make a copy between the two Drive accounts?

    • @intovps said:
      I'm glad you found a solution. I'm curious if Google will let you use 150 Mbit/s constantly.

      Shouldn't be an issue. The daily upload limits are the only restrictions (for now). I've done something similar as this, albeit with not as much data

      And btw, doesn't this work with share + make a copy between the two Drive accounts?

      It does, but that process is still subject to the daily limit and it's much more manually intensive than just running a single command in a terminal window :P

      Thanked by 1intovps

      LowEndStorage | FreeMach - Free KVM Servers (LET Thread)
      Yet Another Bench Script (YABS) - Linux Server Benchmarking Script (dd, iperf, & Geekbench)

    • Also, huge shout out and thank you to @DA_Mark for his generosity and help with this. You're a great person :)

      Thanked by 2ITLabs Ympker

      LowEndStorage | FreeMach - Free KVM Servers (LET Thread)
      Yet Another Bench Script (YABS) - Linux Server Benchmarking Script (dd, iperf, & Geekbench)

    Sign In or Register to comment.