Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Subscribe to our newsletter

Advertise on LowEndTalk.com

Latest LowEndBox Offers

    What is better FTP or Rsync for 200GB mp3 dirctory
    New on LowEndTalk? Please read our 'Community Rules' by clicking on it in the right menu!

    What is better FTP or Rsync for 200GB mp3 dirctory

    I would like to back up a 200GB mp3 directory from one server to another. Which would be faster to do it all in one clip, FTP or Rsync?

    Comments

    • Anything but FTP, unless you are talking about SFTP or FTPS.

    • oops, I meant Sftp, not Ftp.

    • singsingsingsing Member
      edited September 2015

      By default, rsync skips file transfer if size and modtime match on the destination, making it take no more CPU than the other options in simple cases. In complex cases it will save some transfer at the cost of CPU usage.

    • I would suggest rsync as you can resume from failure.
      lftp mirror may achieve the same goal.

      My site, powered by Netlify and Let's Encrypt.

    • bookstack said: I would suggest rsync as you can resume from failure. lftp mirror may achieve the same goal.

      That's the real issue. For a one time transfer the speed is not important as it would be 200GB regardless.

    • If you have a lot of files, don't use ftp. ftp sends each file in a new connection.

    • just tar it all up and use axel. would be done by now.

      Favourite host in general: Ramnode (affiliate link)
      Favourite host for hourly billing/custom ISOs: Vultr ($50 free credit for new accounts, affiliate link)

    • rsync strongly recommended over ftp. Syncing big amount of data is very likely to fail at some point.

    • I've had upto 4x tx rate improvement over plain rsync (over ssh) by using lftp(over sftp/ssh). It was an emergency move on storage plan expiry.

      Admittedly, that was for large multi GB files where lftp's multi-part downloading helped get past the source provider/upstream's annoying per-connection caps. Going from 6MB/s to 22MB/sec definitely helps for transferring 100s of GBs.

      If you have enough space on source disk, tar up the whole payload and try lftp with 3-6 parallel connections. Else, rsync with resume is appropriate.

      Down with the sic_K_ness...
      The doctor might be going K-razy with the low end spirit. 😉

    • rsync is great :)

      (((o(゚▽゚)o))) If privacy is outlawed, only outlaws will have privacy. (((o(゚▽゚)o)))

      ヽ(`Д´)ノ Everyone should run Tor on their idle servers.

    • rsync 100%

      Thanked by 14n0nx

      Peter hostkey.com Dedicated servers and SSD VPS in Russia and NL since 2008.
      Deals: RU/NL SSD VPS 1c/1/5Gb: 6EUR, RU dedicated J1800/4/500Gb - 35EUR

    • I went with Rsync

      Thanked by 14n0nx
    • Rsync. I just transferred 2.3 gb worth of files both by rysnc and sftp. Rsync completed the task in about 90 minutes, whereas sftp had been running for 13 hours and still not complete. Not to mention rsync has incremental backup so if a file already exists at the target location, it will be skipped, saving bandwidth and time.

      Thanked by 14n0nx

      Covfefe: Ask your senators and congressmen to impeach today!

    • singsingsingsing Member
      edited September 2015

      thagoat said: Rsync completed the task in about 90 minutes, whereas sftp had been running for 13 hours and still not complete.

      Let me guess, you were transferring a whole bunch of small files? The difference comes from the fact that sftp waits for each file to complete before starting the next. This limits performance to a few RTT (= round-trip-time) per file, regardless of your ultimate bandwidth transfer capability. Piping tar through ssh avoids this problem if you really don't want to use rsync.

      Thanked by 14n0nx
    Sign In or Register to comment.