New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
tar backup scipt limit cpu usage
hello
iam trying to create a file backup script using tar -cpzf command
the procedure eat a lot of cpu
i tried to use cpulimit and nice
with nice i use this
nice -n 19 tar cpzf .......
then when i am lookin at htop tar command performs ok but gzip takes load to 1.05
with cpulimit i tried this without luck
tar cpzf ....... & cpulimit -e tar -l 30 & cpulimit -e gzip -l 30
can anyone had same probles with cpu usage and wants to help ? Thanks
Comments
tar -zcvf uses only 2% CPU at my VPS with 2 cores @ 2.8ghz
Is there anything else running on the server? The point of nice is that it lets other processes use the CPU; it does not reduce load average. If there's nothing else running on the server, nice will (and will by design) allow the process to use as much CPU as it can.
@Damian yes it is a webserver with 2cores @ 1999ghZ
@corpus
are you sure it's the cpu that causes the load and not the io ?
@camarg how can i check this ?
leave top open and check %us and %wa
if %us is high it's the cpu
if %wa is high it's the io
with
ionice -c 3 tar cpzf .......got same resultslets test yours way
is the %us. it reaches with some peaks 48-68%
i found an intresting article
http://www.clker.com/blog/2008/06/25/ionice-and-daily-backups/
Since it's the cpu that causes the load, it's ok. I wouldn't consider it a problem having a load of 1 on a two core system
yes you are right
after i used
ionice -c2 -n7 nice -n19 tar cpzf .......as the article suggest above %us and %wa never got over 10-15%
Load reaches 1.05 but i think it's ok now
thanks camarg and to the others
get rid of gzip from tar command and use pigz and to compress tar file to speed up times http://vbtechsupport.com/2094/ and http://vbtechsupport.com/1614/
tar gzip vs tar pigz http://vbtechsupport.com/1614/2/ & http://vbtechsupport.com/1614/3/
of course that uses more cpu resources (spikes) but completes faster - just have to find the right balance
That's.. a lot of gigahertz...
Regardless, if it's 2 cores, then I wouldn't worry about it til load is over 2.00.
tar -z passes the compression on to gzip. If you install pigz then you can run the compression via that instead which will spread it out over all (or any number of) CPU cores. This probably won't solve your problem but it'll get the task done faster so it's "out of the way". :-)