Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Rsync out of memory? Try this...
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Rsync out of memory? Try this...

pubcrawlerpubcrawler Banned
edited December 2012 in Tutorials

If you do proper backups and are a shell type person, you probably are an rsync user.

In the lowend market, where there is limited RAM available, rsync can really fail in a big way. Failure is out of memory related when rsync attempts to load all the filenames and info in to RAM at startup.

If you are getting out of memory errors and done the search engine researching and swear rsync has a bug, fear not.

This is your new rsync commandline option to postpend to your current script or cron job:

--no-inc-recursive

That cured our ills on a 256MB VPS. Bound to work just as well on smaller instances.

Thanked by 1cheapdave

Comments

  • pubcrawlerpubcrawler Banned
    edited December 2012

    I run into this issue all the time on smaller RAM ARM boxes. Haven't given it a spin yet on those. The feature was released somewhere in the 3.xxx release timeline, so it's "recent" and doesn't appear in the ancient write ups folks have on rsync from 2005 :)

  • RobertClarkeRobertClarke Member, Host Rep

    Nice tip, will be using that on my new storage box, thanks.

  • Working well on my RPi, big improvement actually.

  • corpuscorpus Member
    edited December 2012

    Incremental recursion can be disabled using the --no-inc-recursive option or its shorter
    --no-i-r alias.

    Nice tip

  • Glad it helps @Andre!

    Thanks @corpus. I like your tag line there! Too bad apt-get won't a beer :)

  • @pubcrawler said: Too bad apt-get won't a beer :)

    It won't if you're not running the command as superuser. (-:

    Relevant xkcd

  • @andre I just had to justify what sudo meant in class... Because I was in piss fits at that.

  • You can also use something like,

    --rsync-path="nice -n 19 /usr/bin/rsync"

    To lower the priority of the resync process on the remote end. Apparently, with a modern kernel nice also adjusts IO priority.

    Of course you'd also nice the local end with,

    nice -n 19 /usr/bin/rsync ...

  • Thanks @pubcrawler, @sleddog. Very nice. :-)

    @Andre - lol

Sign In or Register to comment.