New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Comments
@shovenose
shovehost.com 4GRAM +2G vSwap OpenVZ1 node:
ioping . -c 10 :
--- . (simfs /vz/private/140) ioping statistics ---
10 requests completed in 9513.0 ms, 20 iops, 0.1 mb/s
min/avg/max/mdev = 0.2/51.2/265.5/87.3 ms
ioping . -c 64 :
--- . (simfs /vz/private/140) ioping statistics ---
64 requests completed in 65768.8 ms, 23 iops, 0.1 mb/s
min/avg/max/mdev = 0.2/43.2/1036.6/176.7 ms
ioping . -R :
--- . (simfs /vz/private/140) ioping statistics ---
1158 requests completed in 3000.5 ms, 395 iops, 1.5 mb/s
min/avg/max/mdev = 0.1/2.5/79.6/4.4 ms
ioping . -c 10
--- . (simfs /vz/private/140) ioping statistics ---
10 requests completed in 9051.0 ms, 200 iops, 0.8 mb/s
min/avg/max/mdev = 0.3/5.0/46.6/13.9 ms
ioping . -c 64 :
--- . (simfs /vz/private/140) ioping statistics ---
64 requests completed in 65127.1 ms, 30 iops, 0.1 mb/s
min/avg/max/mdev = 0.2/33.1/939.7/133.1 ms
dd if=/dev/zero of=test bs=64k count=4k oflag=dsync,268435456 bytes (268 MB) copied, 1000.16 s, 268 kB/s
dd if=/dev/zero of=test bs=1M count=1k oflag=dsync,1073741824 bytes (1.1 GB) copied, 165.165 s, 6.5 MB/s
dd if=/dev/zero of=test bs=64k count=16k oflag=dsync,1073741824 bytes (1.1 GB) copied, 3029.71 s, 354 kB/s
dd if=/dev/zero of=test bs=64k count=16k conv=fdatasync,1073741824 bytes (1.1 GB) copied, 25.5921 s, 42.0 MB/s
dd if=/dev/zero of=test bs=1M count=1k conv=fdatasync,1073741824 bytes (1.1 GB) copied, 13.3084 s, 80.7 MB/s
dd if=/dev/zero of=test bs=64k count=4k conv=fdatasync,268435456 bytes (268 MB) copied, 3.57653 s, 75.1 MB/s
shovehost.com 4GRAM +2G vSwap OpenVZ2 node:
ioping . -c 10 :
--- . (simfs /vz/private/137) ioping statistics ---
10 requests completed in 9054.5 ms, 189 iops, 0.7 mb/s
min/avg/max/mdev = 0.4/5.3/10.8/3.0 ms
ioping . -c 64 :
--- . (simfs /vz/private/137) ioping statistics ---
64 requests completed in 63172.3 ms, 390 iops, 1.5 mb/s
min/avg/max/mdev = 0.4/2.6/18.2/3.3 ms
ioping . -R :
--- . (simfs /vz/private/137) ioping statistics ---
810 requests completed in 3000.6 ms, 276 iops, 1.1 mb/s
min/avg/max/mdev = 0.4/3.6/16.9/2.3 ms
dd if=/dev/zero of=test bs=64k count=4k oflag=dsync,268435456 bytes (268 MB) copied, 125.243 s, 2.1 MB/s
dd if=/dev/zero of=test bs=1M count=1k oflag=dsync,1073741824 bytes (1.1 GB) copied, 51.3205 s, 20.9 MB/s
dd if=/dev/zero of=test bs=64k count=16k oflag=dsync,1073741824 bytes (1.1 GB) copied, 528.88 s, 2.0 MB/s
dd if=/dev/zero of=test bs=64k count=16k conv=fdatasync,1073741824 bytes (1.1 GB) copied, 18.1233 s, 59.2 MB/s
dd if=/dev/zero of=test bs=1M count=1k conv=fdatasync,1073741824 bytes (1.1 GB) copied, 19.5088 s, 55.0 MB/s
dd if=/dev/zero of=test bs=64k count=4k conv=fdatasync,268435456 bytes (268 MB) copied, 4.13942 s, 64.8 MB/s
NodeDeploy 1G kvm
ioping . -c 10 :
--- . (ext3 /dev/mapper/nodedeploykvm-root) ioping statistics ---
10 requests completed in 9020.4 ms, 1243 iops, 4.9 mb/s
min/avg/max/mdev = 0.5/0.8/2.3/0.5 ms
ioping . -c 64 :
--- . (ext3 /dev/mapper/nodedeploykvm-root) ioping statistics ---
64 requests completed in 63125.8 ms, 1300 iops, 5.1 mb/s
min/avg/max/mdev = 0.4/0.8/3.0/0.3 ms
ioping . -R :
--- . (ext3 /dev/mapper/nodedeploykvm-root) ioping statistics ---
1463 requests completed in 3000.5 ms, 1686 iops, 6.6 mb/s
min/avg/max/mdev = 0.4/0.6/7.4/0.3 ms
1G OpenVZ
ioping . -c 10 : 10 requests completed in 9080.6 ms, 126 iops, 0.5 mb/s
min/avg/max/mdev = 0.1/8.0/43.5/12.7 ms
ioping . -c 64 : 64 requests completed in 63156.8 ms, 425 iops, 1.7 mb/s
min/avg/max/mdev = 0.1/2.4/44.4/6.7 ms
ioping . -R : 9894 requests completed in 3000.1 ms, 4190 iops, 16.4 mb/s
min/avg/max/mdev = 0.0/0.2/40.7/1.2 ms
BitAccel -
[root@ny1 ioping-0.6]# ./ioping -c 10 .
4096 bytes from . (ext4 /dev/sda3): request=1 time=0.1 ms
4096 bytes from . (ext4 /dev/sda3): request=2 time=0.1 ms
4096 bytes from . (ext4 /dev/sda3): request=3 time=0.1 ms
4096 bytes from . (ext4 /dev/sda3): request=4 time=0.1 ms
4096 bytes from . (ext4 /dev/sda3): request=5 time=0.1 ms
4096 bytes from . (ext4 /dev/sda3): request=6 time=0.1 ms
4096 bytes from . (ext4 /dev/sda3): request=7 time=0.1 ms
4096 bytes from . (ext4 /dev/sda3): request=8 time=0.1 ms
4096 bytes from . (ext4 /dev/sda3): request=9 time=0.1 ms
4096 bytes from . (ext4 /dev/sda3): request=10 time=0.1 ms
--- . (ext4 /dev/sda3) ioping statistics ---
10 requests completed in 9002.2 ms, 7994 iops, 31.2 mb/s
min/avg/max/mdev = 0.1/0.1/0.1/0.0 ms
[root@ny1 ioping-0.6]# dd if=/dev/zero of=test bs=64k count=16k conv=fdatasync
16384+0 records in
16384+0 records out
1073741824 bytes (1.1 GB) copied, 4.53131 s, 237 MB/s
DireVPS.com 4GRAM+1G vSwap
ioping . -c 10 :
--- . (simfs /vps/3130) ioping statistics ---
10 requests completed in 9002.6 ms, 9346 iops, 36.5 mb/s
min/avg/max/mdev = 0.1/0.1/0.1/0.0 ms
ioping . -c 64 :
--- . (simfs /vps/3130) ioping statistics ---
64 requests completed in 63021.5 ms, 8274 iops, 32.3 mb/s
min/avg/max/mdev = 0.1/0.1/0.2/0.0 ms
ioping . -R :
--- . (simfs /vps/3130) ioping statistics ---
23789 requests completed in 3000.1 ms, 15590 iops, 60.9 mb/s
min/avg/max/mdev = 0.0/0.1/4.1/0.0 ms
I replaced my 2 UGVPS VPS's (iops of 14-120, dd 50-140 MB/s depending on time of day), with a BudgetVM Miami openvz. Much better
--- ioping statistics ---
10 requests completed in 9002.3 ms, 9141 iops, 35.7 mb/s
min/avg/max/mdev = 0.1/0.1/0.1/0.0 ms
10 requests completed in 9002.3 ms, 7955 iops, 31.1 mb/s
min/avg/max/mdev = 0.1/0.1/0.2/0.0 ms
10 requests completed in 9002.2 ms, 8489 iops, 33.2 mb/s
min/avg/max/mdev = 0.1/0.1/0.1/0.0 ms
dd
1073741824 bytes (1.1 GB) copied, 2.61076 s, 411 MB/s
edit:
dd if=/dev/zero of=test bs=64k count=16k oflag=dsync,1073741824 bytes (1.1 GB) copied, 3029.71 s, 354 kB/s
10 requests completed in 9513.0 ms, 20 iops, 0.1 mb/s
that box would make a lovely paperweight
Lets specify when it is ovz and when kvm/xen. Something is clearly wrong.
Burst.net 256M RAM 512M Burst OpenVZ
ioping . -c 10 :
--- . (simfs /dev/simfs) ioping statistics ---
10 requests completed in 9012.4 ms, 7605 iops, 29.7 mb/s
min/avg/max/mdev = 0.1/0.1/0.2/0.0 ms
ioping . -c 64 :
--- . (simfs /dev/simfs) ioping statistics ---
64 requests completed in 63260.7 ms, 381 iops, 1.5 mb/s
min/avg/max/mdev = 0.1/2.6/75.5/12.5 ms
ioping . -R :
--- . (simfs /dev/simfs) ioping statistics ---
2955 requests completed in 3000.5 ms, 10289 iops, 40.2 mb/s
min/avg/max/mdev = 0.0/0.1/21.4/0.4 ms
Which examples from this thread do folks truly show a problem with ioping results -- the program, not the VPS it was ran on?
chicagovps.net 2G RAM Enterprise OpenVZ
ioping . -c 10 :
--- . (simfs /vz/private/7477) ioping statistics ---
10 requests completed in 9009.6 ms, 1791 iops, 7.0 mb/s
min/avg/max/mdev = 0.1/0.6/4.7/1.4 ms
ioping . -c 64 :
--- . (simfs /vz/private/7477) ioping statistics ---
64 requests completed in 63016.1 ms, 6837 iops, 26.7 mb/s
min/avg/max/mdev = 0.0/0.1/1.6/0.2 ms
ioping . -R :
--- . (simfs /vz/private/7477) ioping statistics ---
27659 requests completed in 3000.1 ms, 21154 iops, 82.6 mb/s
min/avg/max/mdev = 0.0/0.0/13.1/0.1 ms
Braginhost.net Extreme 2GRAM
ioping . -c 10 :
--- . (simfs /vz/private/723) ioping statistics ---
10 requests completed in 9123.6 ms, 82 iops, 0.3 mb/s
min/avg/max/mdev = 0.3/12.2/38.5/13.7 ms
ioping . -c 64 :
--- . (simfs /vz/private/723) ioping statistics ---
64 requests completed in 64728.1 ms, 37 iops, 0.1 mb/s
min/avg/max/mdev = 0.2/26.9/518.1/74.3 ms
ioping . -R :
--- . (simfs /vz/private/723) ioping statistics ---
392 requests completed in 3010.0 ms, 132 iops, 0.5 mb/s
min/avg/max/mdev = 0.2/7.6/155.9/11.9 ms
Well, just the post I quoted from Zen.
I also checked an empty ssd node and got 100 or so IOPS. Not in a container, directly on the node, not to mention wild variations on an empty node.
SSDs entering power saving mode when idle or something?
But people should not overhype about these benchmarks. Benchmarks are artificial load, artificial access patterns. The real life is different from the benchmarks.
@Maounique,
Saw Zen's results. Curious indeed. Broken? Unsure. Seems like something could be different with SSDs though.
@Zen I'm not cleaning up your throwup.
Just tested on RAID1 empty server, one time I had over 60k iops, other time I had 10.
@Alex_LiquidHost, 60k, sounds more like it. 10 though, very strange.
Perhaps in another terminal you can run iostat and eyeball the activity and see if anything going on there during entire test time and compare activity during both tests.
@Zen I use AMD Opteron or Intel Core/Xeon
60k for raid 1 ? Unlikely.
I second that.
10 iops for empty server with RE4's is unlikely as well
Yes, we saw many unlikely on the lower end of the spectrum, on the high side without knowing what storage is on the machine, you cant easily tell, but in your case, we knew, so it errs in both directions.
Certainly something with the SSDs going on. Wondering if folks are all seeing some artifact from the RAID controller. Anyone have bare SSD to test minus a fancy card? Process of elimination.
OpenVZ - RHEL6 - EXT3 FS (RAID01)
KVM Node - RHEL6 - EXT3 FS (RAID10)
KVM Node - RHEL6 - EXT4 FS (RAID10 w/ BBU)
Looks like BBU makes a lot of difference, but im going to say this test is probably as accurate as DD.
Not even. dd does not vary to such extent, from 60k to 10 in an empty node ? No dd will go from 600 MB/s to 10 KB/s.
And I think that the BBU unit has write cache. We are close to narrowing the problem down. I think it depends how free is the cache and that can vary fast.
I dont know if Alex had hw raid ?
I'm getting a new node right now (openvz3) that should be better. All new orders will randomly go to openvz1, openvz2, or openvz3.
it will be better.
dd is consistent in my experience.
The past 5 days, I am having problems with one of our hostnodes, performing extremely bad, so I've probably ran over 100 dd's in different enviroments (test and production) and lots of iops tests. The dd tests are consistent, the iops once are just hit or miss.
As far as I know a decent raid controller with BBU should output much, much more than other raid arrays without BBU/writecache.
I don't thing anywhere in this thread that the spinning drives have been proven to have strange numbers like the SSDs. But, plausible also.
And I think that the BBU unit has write cache. We are close to narrowing the problem down. I think it depends how free is the cache and that can vary fast.
I dont know if Alex had hw raid ?
I'm starting to think SW RAID can cause those crazy jumps in numbers as well.
There is nothing strange. Ioping writes a 64mb file and then does random reading from the file, dropping OS caches between the reads.
Modern HDDs have 64MB buffer so if the data alignment is right and there is not other IO happening during the ioping tests all the reads can come from the HDD cache without need to read from the platters. This is when you see the big numbers (60k iops) - it is reading from the HDD memory buffer.
Or in the case of a RAID controller with a buffer - it is reading from the RAID controller memory.
And something else - for a KVM VPS you can sometimes see higher iops inside the VPS than on the node itself. The reason is that the ioping can instruct the VPS kernel to drop caches, but the data is still cached on the node. So when the data is read back it is coming from the memory cache on the node, not from disk.
Not sure with the drive/controller caching and various virtualization software how much this will matter:
-C use cached I/O
-D use direct I/O
Anyone with funky results want to run ioping with those added on two different tests and see if anything remotely different happens?