First impressions NVMe KVM first-root.com
as there were some changes to the offers from providers I tend to use (hetzner vs. netcup ;-)) and all the NVMe related discussions lately, I had another look around the german market to see what providers are making steps towards NVMe...
I stumbled across the offers of https://www.first-root.com/ - who are in the german market for quite some time now (I think from 2011 or so) but never made it to my list of providers that are of interest to me. can't remember why that is, most likely because not really low end and quite expensive for what I usually look out...
yet if one does believe that NVMe could be a game changer, offers with large disk of that kind are still not easy to find - and that is something First Root offers. while investigating I came in contact with their CEO @FR_Michael who is now lurking around here and interestingly enough we found that we played quite some rounds of CS together back in the days ^^
he offered me a test drive of their NVMe products which I took ofc, because always love to have something new to play with. so without further ado, benchmarks...
someone lately mentioned VPS reviews doing a php compilation, so I did that too - it took 3m49.251s.
they are hosting out of KPN DC in Dusseldorf using them as part of their upstream but have their own equipment and AS (as41108), serverbench (incoming wget network speeds):
# bash serverbench.sh ---------------------------------------------------------------------- CPU model : Intel(R) Core(TM) i7-4770T CPU @ 2.50GHz Number of cores : 2 CPU frequency : 2494.224 MHz Total amount of ram : 3859 MB Total amount of swap : 2282 MB System uptime : 1days, 17:28:55 Load average : 0.00, 0.09, 0.22 OS : Debian GNU/Linux 9 Arch : x86_64 (64 Bit) Kernel : 4.9.0-5-amd64 ---------------------------------------------------------------------- Node Name IPv4 address Download Speed CacheFly 220.127.116.11 88.6MB/s Vultr, Tokyo, JP 18.104.22.168 8.59MB/s Linode, Tokyo, JP 22.214.171.124 9.76MB/s DO, Bangalore, IN 126.96.36.199 12.2MB/s Softlayer, Chennai, IN 188.8.131.52 5.38MB/s Vultr, Singapore, SG 184.108.40.206 8.96MB/s DO, Singapore, SG 220.127.116.11 6.65MB/s Linode, Singapore, SG 18.104.22.168 6.61MB/s Softlayer, Singapore, SG 22.214.171.124 5.14MB/s Leaseweb, Singapore, SG 126.96.36.199 8.76MB/s Softlayer, HongKong, CN 188.8.131.52 4.90MB/s Leaseweb, HongKong, CN 184.108.40.206 7.87MB/s Vultr, Sydney, AUS 220.127.116.11 6.98MB/s Softlayer, Sydney, AUS 18.104.22.168 3.67MB/s Softlayer, Melbourne, AUS 22.214.171.124 3.82MB/s Tele2, Gothenberg, SE 126.96.36.199 34.7MB/s Tele2, Kista, SE 188.8.131.52 27.4MB/s Softlayer, Milan, IT 184.108.40.206 50.3MB/s Prometeus, Milan, IT 220.127.116.11 45.9MB/s Tele2, Riga, LV 18.104.22.168 27.4MB/s Tele2, Vilnius, LT 22.214.171.124 26.5MB/s Server.LU, Luxembourg, LU 126.96.36.199 60.3MB/s Tele2, Frankfurt, DE 188.8.131.52 70.2MB/s Vultr, Frankfurt, DE 184.108.40.206 99.7MB/s Linode, Frankfurt, DE 220.127.116.11 103MB/s Softlayer, Frankfurt, DE 18.104.22.168 37.4MB/s Leaseweb, Frankfurt, DE 22.214.171.124 93.5MB/s DO, Frankfurt, DE 126.96.36.199 103MB/s Vultr, Paris, FR 188.8.131.52 92.4MB/s OVH, Gravelines, FR 184.108.40.206 20.7MB/s OVH, Strasbourg, FR 220.127.116.11 31.7MB/s OVH, Roubaix, FR 18.104.22.168 57.3MB/s Online.Net, Paris, FR 22.214.171.124 57.7MB/s Tele2, Amsterdam, NL 126.96.36.199 69.3MB/s Vultr, Amsterdam, NL 188.8.131.52 103MB/s DO 2, Amsterdam, NL 184.108.40.206 98.3MB/s DO 3, Amsterdam, NL 220.127.116.11 103MB/s Leaseweb, Amsterdam, NL 18.104.22.168 98.0MB/s i3d, Amsterdam, NL 22.214.171.124 91.9MB/s Vultr, London, UK 126.96.36.199 98.1MB/s DO, London, UK 188.8.131.52 99.0MB/s Linode, London, UK 184.108.40.206 87.8MB/s Softlayer, London, UK 220.127.116.11 60.2MB/s Softlayer, Mexico, MX 18.104.22.168 8.37MB/s Softlayer, Brazil, BR 22.214.171.124 5.94MB/s DO 1, NYC, USA 126.96.36.199 27.6MB/s DO 2, NYC, USA 188.8.131.52 26.4MB/s DO 3, NYC, USA 184.108.40.206 26.4MB/s Vultr, New Jersey, USA 220.127.116.11 27.9MB/s Linode, Newark, USA 18.104.22.168 25.5MB/s Vultr, Illinois, USA 22.214.171.124 24.2MB/s Vultr, Atlanta, USA 126.96.36.199 23.7MB/s Linode, Atlanta, USA 188.8.131.52 20.3MB/s Vultr, Miami, USA 184.108.40.206 21.2MB/s Vultr, Washington, USA 220.127.116.11 15.8MB/s Softlayer, Seattle, USA 18.104.22.168 7.93MB/s Leaseweb, Washington, USA 22.214.171.124 21.0MB/s Vultr, Dallas, USA 126.96.36.199 18.6MB/s Linode, Dallas, USA 188.8.131.52 16.0MB/s Softlayer, Dallas, USA 184.108.40.206 10.3MB/s Leaseweb, Dallas, USA 220.127.116.11 16.4MB/s Vultr, Los Angeles, USA 18.104.22.168 14.9MB/s DO, San Francisco, USA 22.214.171.124 14.2MB/s DO, San Francisco, USA 126.96.36.199 14.1MB/s Linode, Fremont, USA 188.8.131.52 11.3MB/s Leaseweb, San Francisco, USA 184.108.40.206 14.7MB/s DO, Toronto, CA 220.127.116.11 25.2MB/s OVH, Beauharnois, CA 18.104.22.168 2.67MB/s EastLink, Canada, CA 22.214.171.124 7.44MB/s Softlayer, Montreal, CA 126.96.36.199 13.6MB/s ---------------------------------------------------------------------- I/O speed(1st run) : 1.1 GB/s I/O speed(2nd run) : 1.4 GB/s I/O speed(3rd run) : 1.3 GB/s Average I/O speed : 1297.07 MB/s
now for the NVMe, that box comes with whopping 200GB of it...
# ioping -c 10 . --- . (ext4 /dev/vda1) ioping statistics --- 9 requests completed in 2.96 ms, 36 KiB read, 3.04 k iops, 11.9 MiB/s generated 10 requests in 9.00 s, 40 KiB, 1 iops, 4.44 KiB/s min/avg/max/mdev = 227.5 us / 328.4 us / 541.5 us / 82.4 us
# fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=32 --size=4G --numjobs=4 --readwrite=randrw --rwmixread=50 test: (groupid=0, jobs=1): err= 0: pid=26134: Sun Feb 18 15:58:23 2018 read : io=2046.9MB, bw=42626KB/s, iops=10656, runt= 49172msec write: io=2049.2MB, bw=42673KB/s, iops=10668, runt= 49172msec cpu : usr=5.74%, sys=14.03%, ctx=751967, majf=0, minf=10 test: (groupid=0, jobs=1): err= 0: pid=26135: Sun Feb 18 15:58:23 2018 read : io=2048.2MB, bw=42683KB/s, iops=10670, runt= 49136msec write: io=2047.1MB, bw=42678KB/s, iops=10669, runt= 49136msec cpu : usr=4.57%, sys=14.03%, ctx=761987, majf=0, minf=8 test: (groupid=0, jobs=1): err= 0: pid=26136: Sun Feb 18 15:58:23 2018 read : io=2046.8MB, bw=42616KB/s, iops=10654, runt= 49164msec write: io=2049.1MB, bw=42696KB/s, iops=10674, runt= 49164msec cpu : usr=5.15%, sys=14.08%, ctx=749131, majf=0, minf=8 test: (groupid=0, jobs=1): err= 0: pid=26137: Sun Feb 18 15:58:23 2018 read : io=2049.5MB, bw=42744KB/s, iops=10685, runt= 49099msec write: io=2046.6MB, bw=42682KB/s, iops=10670, runt= 49099msec cpu : usr=4.96%, sys=14.81%, ctx=777138, majf=0, minf=8 Run status group 0 (all jobs): READ: io=8190.6MB, aggrb=170566KB/s, minb=42616KB/s, maxb=42743KB/s, mint=49099msec, maxt=49172msec WRITE: io=8193.5MB, aggrb=170627KB/s, minb=42672KB/s, maxb=42696KB/s, mint=49099msec, maxt=49172msec Disk stats (read/write): vda: ios=2095744/2096523, merge=0/10, ticks=3225128/2943024, in_queue=6192932, util=100.00%
# fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=64k --iodepth=32 --size=4G --numjobs=4 --readwrite=randrw --rwmixread=50 test: (groupid=0, jobs=1): err= 0: pid=26122: Sun Feb 18 15:50:49 2018 read : io=2043.3MB, bw=186279KB/s, iops=2910, runt= 11232msec write: io=2052.8MB, bw=187145KB/s, iops=2924, runt= 11232msec cpu : usr=2.32%, sys=7.27%, ctx=58959, majf=0, minf=8 test: (groupid=0, jobs=1): err= 0: pid=26123: Sun Feb 18 15:50:49 2018 read : io=2059.2MB, bw=187393KB/s, iops=2928, runt= 11252msec write: io=2036.9MB, bw=185368KB/s, iops=2896, runt= 11252msec cpu : usr=3.02%, sys=6.43%, ctx=58861, majf=0, minf=6 test: (groupid=0, jobs=1): err= 0: pid=26124: Sun Feb 18 15:50:49 2018 read : io=2051.8MB, bw=186616KB/s, iops=2915, runt= 11258msec write: io=2044.4MB, bw=185946KB/s, iops=2905, runt= 11258msec cpu : usr=2.45%, sys=6.96%, ctx=58946, majf=0, minf=6 test: (groupid=0, jobs=1): err= 0: pid=26125: Sun Feb 18 15:50:49 2018 read : io=2033.1MB, bw=185662KB/s, iops=2900, runt= 11218msec write: io=2062.7MB, bw=188229KB/s, iops=2941, runt= 11218msec cpu : usr=2.39%, sys=7.10%, ctx=58789, majf=0, minf=6 Run status group 0 (all jobs): READ: io=8188.0MB, aggrb=744760KB/s, minb=185661KB/s, maxb=187392KB/s, mint=11218msec, maxt=11258msec WRITE: io=8196.0MB, aggrb=745488KB/s, minb=185367KB/s, maxb=188228KB/s, mint=11218msec, maxt=11258msec Disk stats (read/write): vda: ios=129883/130015, merge=0/2, ticks=842376/579484, in_queue=1424276, util=99.36%
to be able to compare those to other NVMe, I spun up a 4G instance at Hetzner FSN:
# fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=32 --size=4G --numjobs=4 --readwrite=randrw --rwmixread=50 test: (groupid=0, jobs=1): err= 0: pid=1373: Sun Feb 18 16:51:06 2018 read : io=2046.9MB, bw=29950KB/s, iops=7487, runt= 69983msec write: io=2049.2MB, bw=29983KB/s, iops=7495, runt= 69983msec cpu : usr=4.74%, sys=26.25%, ctx=629695, majf=0, minf=11 test: (groupid=0, jobs=1): err= 0: pid=1374: Sun Feb 18 16:51:06 2018 read : io=2048.2MB, bw=30082KB/s, iops=7520, runt= 69719msec write: io=2047.1MB, bw=30079KB/s, iops=7519, runt= 69719msec cpu : usr=4.97%, sys=26.20%, ctx=627934, majf=0, minf=9 test: (groupid=0, jobs=1): err= 0: pid=1375: Sun Feb 18 16:51:06 2018 read : io=2046.8MB, bw=30125KB/s, iops=7531, runt= 69550msec write: io=2049.1MB, bw=30182KB/s, iops=7545, runt= 69550msec cpu : usr=4.69%, sys=26.31%, ctx=626929, majf=0, minf=8 test: (groupid=0, jobs=1): err= 0: pid=1376: Sun Feb 18 16:51:06 2018 read : io=2049.5MB, bw=29974KB/s, iops=7493, runt= 70016msec write: io=2046.6MB, bw=29931KB/s, iops=7482, runt= 70016msec cpu : usr=4.55%, sys=26.40%, ctx=629930, majf=0, minf=8 Run status group 0 (all jobs): READ: io=8190.6MB, aggrb=119788KB/s, minb=29950KB/s, maxb=30124KB/s, mint=69550msec, maxt=70016msec WRITE: io=8193.5MB, aggrb=119831KB/s, minb=29930KB/s, maxb=30181KB/s, mint=69550msec, maxt=70016msec Disk stats (read/write): sda: ios=2085959/2086492, merge=0/49, ticks=1915696/1479564, in_queue=3399560, util=100.00%
# fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=64k --iodepth=32 --size=4G --numjobs=4 --readwrite=randrw --rwmixread=50 test: (groupid=0, jobs=1): err= 0: pid=1384: Sun Feb 18 16:53:38 2018 read : io=2043.3MB, bw=90103KB/s, iops=1407, runt= 23221msec write: io=2052.8MB, bw=90522KB/s, iops=1414, runt= 23221msec cpu : usr=2.64%, sys=7.42%, ctx=28913, majf=0, minf=9 test: (groupid=0, jobs=1): err= 0: pid=1385: Sun Feb 18 16:53:38 2018 read : io=2059.2MB, bw=90600KB/s, iops=1415, runt= 23273msec write: io=2036.9MB, bw=89621KB/s, iops=1400, runt= 23273msec cpu : usr=2.49%, sys=8.06%, ctx=29278, majf=0, minf=9 test: (groupid=0, jobs=1): err= 0: pid=1386: Sun Feb 18 16:53:38 2018 read : io=2051.8MB, bw=90262KB/s, iops=1410, runt= 23276msec write: io=2044.4MB, bw=89937KB/s, iops=1405, runt= 23276msec cpu : usr=2.23%, sys=7.99%, ctx=29405, majf=0, minf=7 test: (groupid=0, jobs=1): err= 0: pid=1387: Sun Feb 18 16:53:38 2018 read : io=2033.1MB, bw=89488KB/s, iops=1398, runt= 23274msec write: io=2062.7MB, bw=90726KB/s, iops=1417, runt= 23274msec cpu : usr=2.03%, sys=8.08%, ctx=29457, majf=0, minf=7 Run status group 0 (all jobs): READ: io=8188.0MB, aggrb=360221KB/s, minb=89488KB/s, maxb=90600KB/s, mint=23221msec, maxt=23276msec WRITE: io=8196.0MB, aggrb=360573KB/s, minb=89621KB/s, maxb=90725KB/s, mint=23221msec, maxt=23276msec Disk stats (read/write): sda: ios=130959/131106, merge=0/4, ticks=1566644/1183516, in_queue=2757732, util=99.74%
so they easily beat hetzner IO wise with 42k vs. 30k on the 4k blocksize and 11.6k vs 5.6k with 64k blocksize (50% readwrite mix) - I have been told the node is filled about 75% but it is dedicated ressources in this price category anyway.
I am not using other suspects like DO or Ramnode, so can't tell about the numbers there, but I'd say NVMe clearly makes a point here ;-)
other facts regarding First Root: CPU is passthrough, so VME/VMX is available as is AES-NI. mining related stuff is forbidden by their TOS (for sole benchmarking purposes: xmrig achieved around 140H/s out of those two 4770T cores :-P)
I set up proxmox on that box and everything worked fine as nested virt (LXC/KVM), no issues - not many providers offer that.
also a single addon IP is 1€ (without VAT), VLAN is available for free in combination with one or more subnets.
their control panel is nice and clean and also available in english language (one might need to re-login). VNC is available, support seems helpful if you need additional images (I am fine with debian, thx). afaik byol windows is no problem, though I did not see images for installing that ;-)
at least with their Ultra packages they do include a backup which is taken on the fly (without downtime) and not counted towards the disk space but placed on extra infrastructure, very nice feature!
for what it's worth, you might to watch out for the usual german habits like contract terms and 1 month cancellation period (cancellation can be done easily from within the control panel)
things are running very smoothly now for more than 2 weeks, so overall for a first impression a very interesting product lineup.
most likely not for cheapskates, as it's competing in the range of bigger VMs like from ramnode, buyvm, and maybe DO, vultr, hetzner - even though no hourly cloud stuff yet.
as they also have some smaller product ranges starting from around € (incl. german VAT, which they deduct for international customers), maybe @FR_Michael can be convinced to release some offers/special discount for the LET crowd within our pricing range ;-)
edit: changed link from first-root.de to first-root.com to directly go to their international pages ;-)