Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Some more storage and NVMe goodness - Amsterdam - 40G networking - Page 3
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Some more storage and NVMe goodness - Amsterdam - 40G networking

13567

Comments

  • skorousskorous Member

    @TimboJones said:
    What is poor IO performance to you? My Chicago HH servers have some of the highest IO out of more than a dozen VPS's I have.

    Well this thread is about Amsterdam so he's probably referring to one of the 2T boxes there?

  • Daniel15Daniel15 Veteran

    @TimboJones said:

    @TeoM said:
    Thanks to you torrent users I have poor IO performance.

    What is poor IO performance to you? My Chicago HH servers have some of the highest IO out of more than a dozen VPS's I have.

    In my experience their Chicago servers have much better IO performance than some other locations. Haven't tried Amsterdam, but Los Angeles is around 3x slower than Chicago.

  • sanvitsanvit Member
    edited May 2021

    @Daniel15 said: Los Angeles is around 3x slower than Chicago

    Well, just did a YABS for both CHI 10TB and LA 10TB, and Chicago had noticeably lower IOPS for me

    root@la-10t:~# wget -qO- yabs.sh | bash
    # ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
    #              Yet-Another-Bench-Script              #
    #                     v2020-12-29                    #
    # https://github.com/masonr/yet-another-bench-script #
    # ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
    
    Fri May  7 05:39:47 UTC 2021
    
    Basic System Information:
    ---------------------------------
    Processor  : Intel(R) Xeon(R) CPU E5-2690 v2 @ 3.00GHz
    CPU cores  : 2 @ 2999.998 MHz
    AES-NI     : ✔ Enabled
    VM-x/AMD-V : ❌ Disabled
    RAM        : 1.9 GiB
    Swap       : 2.0 GiB
    Disk       : 9.7 TiB
    
    fio Disk Speed Tests (Mixed R/W 50/50):
    ---------------------------------
    Block Size | 4k            (IOPS) | 64k           (IOPS)
      ------   | ---            ----  | ----           ----
    Read       | 28.23 MB/s    (7.0k) | 378.92 MB/s   (5.9k)
    Write      | 28.25 MB/s    (7.0k) | 380.91 MB/s   (5.9k)
    Total      | 56.49 MB/s   (14.1k) | 759.83 MB/s  (11.8k)
               |                      |
    Block Size | 512k          (IOPS) | 1m            (IOPS)
      ------   | ---            ----  | ----           ----
    Read       | 766.84 MB/s   (1.4k) | 837.62 MB/s    (817)
    Write      | 807.59 MB/s   (1.5k) | 893.41 MB/s    (872)
    Total      | 1.57 GB/s     (3.0k) | 1.73 GB/s     (1.6k)
    

    root@chi-10t:~# wget -qO- yabs.sh | bash
    # ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
    #              Yet-Another-Bench-Script              #
    #                     v2020-12-29                    #
    # https://github.com/masonr/yet-another-bench-script #
    # ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
    
    Fri May  7 05:39:49 UTC 2021
    
    Basic System Information:
    ---------------------------------
    Processor  : Intel(R) Xeon(R) CPU E5-2690 v2 @ 3.00GHz
    CPU cores  : 2 @ 2999.998 MHz
    AES-NI     : ✔ Enabled
    VM-x/AMD-V : ❌ Disabled
    RAM        : 1.9 GiB
    Swap       : 2.0 GiB
    Disk       : 9.7 TiB
    
    fio Disk Speed Tests (Mixed R/W 50/50):
    ---------------------------------
    Block Size | 4k            (IOPS) | 64k           (IOPS)
      ------   | ---            ----  | ----           ----
    Read       | 8.86 MB/s     (2.2k) | 178.63 MB/s   (2.7k)
    Write      | 8.90 MB/s     (2.2k) | 179.57 MB/s   (2.8k)
    Total      | 17.76 MB/s    (4.4k) | 358.21 MB/s   (5.5k)
               |                      |
    Block Size | 512k          (IOPS) | 1m            (IOPS)
      ------   | ---            ----  | ----           ----
    Read       | 750.50 MB/s   (1.4k) | 865.85 MB/s    (845)
    Write      | 790.38 MB/s   (1.5k) | 923.52 MB/s    (901)
    Total      | 1.54 GB/s     (3.0k) | 1.78 GB/s     (1.7k)
    
  • Daniel15Daniel15 Veteran
    edited May 2021

    @sanvit said:

    @Daniel15 said: Los Angeles is around 3x slower than Chicago

    Well, just did a YABS for both CHI 10TB and LA 10TB, and Chicago had noticeably lower IOPS for me

    Oooof it's way better than mine :/

    This was mine last month:

    # ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
    #              Yet-Another-Bench-Script              #
    #                     v2020-12-29                    #
    # https://github.com/masonr/yet-another-bench-script #
    # ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
    
    Tue 30 Mar 2021 11:48:01 AM PDT
    Basic System Information:
    ---------------------------------
    Processor  : Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz
    CPU cores  : 2 @ 2799.998 MHz
    AES-NI     : ✔ Enabled
    VM-x/AMD-V : ❌ Disabled
    RAM        : 1.9 GiB
    Swap       : 1.9 GiB
    Disk       : 9.7 TiB
    
    fio Disk Speed Tests (Mixed R/W 50/50):
    ---------------------------------
    Block Size | 4k            (IOPS) | 64k           (IOPS)
      ------   | ---            ----  | ----           ----
    Read       | 4.76 MB/s     (1.1k) | 83.62 MB/s    (1.3k)
    Write      | 4.78 MB/s     (1.1k) | 84.06 MB/s    (1.3k)
    Total      | 9.55 MB/s     (2.3k) | 167.69 MB/s   (2.6k)
               |                      |
    Block Size | 512k          (IOPS) | 1m            (IOPS)
      ------   | ---            ----  | ----           ----
    Read       | 156.77 MB/s    (306) | 189.14 MB/s    (184)
    Write      | 165.10 MB/s    (322) | 201.74 MB/s    (197)
    Total      | 321.87 MB/s    (628) | 390.89 MB/s    (381)
    

    and this was a few days ago:

    fio Disk Speed Tests (Mixed R/W 50/50):
    ---------------------------------
    Block Size | 4k (IOPS) | 64k (IOPS)
    ------ | --- ---- | ---- ----
    Read | 587.00 KB/s (146) | 7.72 MB/s (120)
    Write | 617.00 KB/s (154) | 8.12 MB/s (126)
    Total | 1.20 MB/s (300) | 15.85 MB/s (246)
    | |
    Block Size | 512k (IOPS) | 1m (IOPS)
    ------ | --- ---- | ---- ----
    Read | 15.68 MB/s (30) | 17.72 MB/s (17)
    Write | 17.08 MB/s (33) | 19.78 MB/s (19)
    Total | 32.77 MB/s (63) | 37.51 MB/s (36)
    

    (sorry for the formatting - I lost the spaces when I copied it into a support ticket)

    with lots of iowait when I was copying a ~100MB file across the network: https://d.sb/2021/05/firefox_04-11.13.24.png

    I've got a support ticket open but they didn't see any iowait on the node.

    You do get good value for the price, and their NVMe VPSes have consistent performance (would definitely recommend them as one of my top providers for NVMe VPSes), but their storage VPSes are extremely variable in terms of disk IO performance.

  • sanvitsanvit Member

    @Daniel15 said: but their storage VPSes are extremely variable in terms of disk IO performance.

    looking at your bench, that does seems to be true.
    TBH these are mostly for cold backups anyway, but yeah that speed sucks :(

  • FalzoFalzo Member

    @Daniel15 said: but their storage VPSes are extremely variable in terms of disk IO performance

    probably have to deal with abuse a lot and it takes time to catch those. torrenters, plotters :'(

    Thanked by 2miu bulbasaur
  • @hosthatch invoice #234898 add pls +10 TB bandwidth and +10 GB disk

  • kalipuskalipus Member

    if someone want to trade 2tb servers please pm me, I don't care much about performance

  • miumiu Member
    edited May 2021

    @Falzo said:

    @Daniel15 said: but their storage VPSes are extremely variable in terms of disk IO performance

    probably have to deal with abuse a lot and it takes time to catch those. torrenters, plotters :'(

    Or I/O degradation can be also temporary - in time when RAID Controller currently/regularly re-checks array consistency

    But sure in other cases, all I/O abusers and worms (who abuse storage node for high I/O applications) should be catch and suspended, they harm all - provider and all users also

  • fragpicfragpic Member

    @kalipus said:
    if someone want to trade 2tb servers please pm me, I don't care much about performance

    trade with what?

  • skorousskorous Member

    @fragpic said:

    @kalipus said:
    if someone want to trade 2tb servers please pm me, I don't care much about performance

    trade with what?

    Moot point since transfers are currently unavailable.

    Thanked by 1webcraft
  • PieHasBeenEatenPieHasBeenEaten Member, Host Rep
    edited May 2021

    My experience so far has been fair. Just trying to get one of the interal ips working on my compute vm so i can use the storage server as nfs server. But the replies have been with in a couple hrs but had to bump the ticket since it they last replied 15hrs ago.

  • @skorous said:

    @TimboJones said:
    What is poor IO performance to you? My Chicago HH servers have some of the highest IO out of more than a dozen VPS's I have.

    Well this thread is about Amsterdam so he's probably referring to one of the 2T boxes there?

    Of course. That's why I mentioned my Chicago server so I didn't confuse with another location. But if you don't think torrenting also happens in Chicago or any location, then that's on you. What I don't know is what he considers "poor", since that isn't a standard metric. Perhaps I missed where he provided results.

  • sanvitsanvit Member

    @kalipus said:
    if someone want to trade 2tb servers please pm me, I don't care much about performance

    iirc they don’t allow transfers atm?

  • pierrepierre Member

    Any idea when you'll roll out IPv6? I'm interested but want to have IPv6 support.

    Looks like a great deal!

  • tetechtetech Member

    @pierre said:
    Any idea when you'll roll out IPv6? I'm interested but want to have IPv6 support.

    Looks like a great deal!

    Huh? They support IPv6 already.

  • Daniel15Daniel15 Veteran

    @pierre said:
    Any idea when you'll roll out IPv6? I'm interested but want to have IPv6 support.

    All their locations support IPv6, and I think it's a /64 range in most locations. You just need to open a ticket to request that it be enabled.

  • fazarfazar Member
    edited May 2021

    @pierre said:
    Any idea when you'll roll out IPv6? I'm interested but want to have IPv6 support.

    Looks like a great deal!

    their IPv6 connectivity also very good.

    iperf3 Network Speed Tests (IPv6):

    Provider | Location (Link) | Send Speed | Recv Speed
    | | |
    Clouvider | London, UK (10G) | 8.20 Gbits/sec | 4.02 Gbits/sec
    Online.net | Paris, FR (10G) | 5.27 Gbits/sec | 5.90 Gbits/sec
    WorldStream | The Netherlands (10G) | 12.0 Gbits/sec | 3.70 Gbits/sec
    Clouvider | NYC, NY, US (10G) | 2.52 Gbits/sec | 1.90 Gbits/sec
    Clouvider | Los Angeles, CA, US (10G) | 1.31 Gbits/sec | 1.22 Gbits/sec

    ps. cant get code formatting done on mobile..

  • weaselweasel Member
    edited May 2021

    Impressed with the service so far. Signed up for the 2TB storage deal.

    @pierre said:
    Any idea when you'll roll out IPv6? I'm interested but want to have IPv6 support.

    All their locations support IPv6, and I think it's a /64 range in most locations. You just need to open a ticket to request that it be enabled.

    That's right. I can confirm they have IPv6 on these servers, and it's /64 range. I had to do the following:

    1. Open a ticket requesting IPv6 support (got a response within 2hrs)
    2. Add a new IP in the Servers > Network section
    3. Hit 'Reconfigure Network' (I know you can do it by editing /etc/network/interfaces, but this was easiest for me)> @Daniel15 said:
  • vyas11vyas11 Member
    edited May 2021

    Thx for speed results. You can use the

    “Pre” 

    tag to format on mobile.
    Someone here told me this a while back, passing it on !

    @fazar said:

    Looks like a great deal!

    their IPv6 connectivity also very good.

     iperf3 Network Speed Tests (IPv6):
    ---------------------------------
     Provider        | Location (Link)           | Send Speed      | Recv Speed
                 |                           |                 |
    Clouvider       | London, UK (10G)          | 8.20 Gbits/sec  | 4.02 Gbits/sec
    Online.net      | Paris, FR (10G)           | 5.27 Gbits/sec  | 5.90 Gbits/sec
    WorldStream     | The Netherlands (10G)     | 12.0 Gbits/sec  | 3.70 Gbits/sec
    Clouvider       | NYC, NY, US (10G)         | 2.52 Gbits/sec  | 1.90 Gbits/sec
    Clouvider       | Los Angeles, CA, US (10G) | 1.31 Gbits/sec  | 1.22 Gbits/sec
    
    Thanked by 1fazar
  • hosthatchhosthatch Patron Provider, Top Host, Veteran

    @codelock said:
    How do you deal with DMCA in ams? Want to just use some public torrent trackers

    Also bandwidth is counted for upload +download or only upload?

    We follow copyright laws in all of our locations. There are other providers better suited to this sort of usage :)

    Bandwidth is counted both ways.

    If you run private torrents, it is likely that we will never know and everything will be fine, but any abusive usage likely will get you throttled hard, so it will be a better idea to use providers more suited to this usage.

    @TeoM said:
    Thanks to you torrent users I have poor IO performance.

    Do you mean because of torrent users you might get poor performance in the future? There is no poor IO performance in Amsterdam anywhere at the moment. If you meant that you are seeing poor performance already, please open a ticket.

    @TeoM said:
    @hosthatch How fast is actually the internal connection between the nvme and storage servers ? I reach a maximum of 400MBit. I assumed to get more than 1 Gigabit ?

    Depends on the way you're testing it. Did you try iperf? Here's an iperf I just did on 2 VMs that I spun up (one storage, one NVMe). With more cores, you can reach ~25 Gbps.

    Accepted connection from 10.115.3.21, port 50787
    [  5] local 10.114.1.7 port 5201 connected to 10.115.3.21 port 48441
    [ ID] Interval           Transfer     Bitrate
    [  5]   0.00-1.00   sec  1.43 GBytes  12.3 Gbits/sec
    [  5]   1.00-2.00   sec  1.71 GBytes  14.7 Gbits/sec
    [  5]   2.00-3.00   sec  1.37 GBytes  11.8 Gbits/sec
    [  5]   3.00-4.00   sec  1.32 GBytes  11.4 Gbits/sec
    [  5]   4.00-5.00   sec  1.52 GBytes  13.1 Gbits/sec
    [  5]   5.00-6.00   sec  1.36 GBytes  11.7 Gbits/sec
    [  5]   6.00-7.00   sec  1.28 GBytes  11.0 Gbits/sec
    [  5]   7.00-8.00   sec  1.38 GBytes  11.8 Gbits/sec
    [  5]   8.00-9.00   sec  1.65 GBytes  14.2 Gbits/sec
    [  5]   9.00-10.00  sec  1.36 GBytes  11.7 Gbits/sec
    [  5]  10.00-10.04  sec  66.1 MBytes  14.2 Gbits/sec
    - - - - - - - - - - - - - - - - - - - - - - - - -
    [ ID] Interval           Transfer     Bitrate
    [  5]   0.00-10.04  sec  14.4 GBytes  12.4 Gbits/sec                  receiver
    

    @lianpai said:
    Do you have plan to provider storage server on HK? @hosthatch

    Our costs in HK are about 4 times higher - so it will have to be 4x expensive. We're not one of those HE/Cogent-only providers, since that is the only way you can make things cheap in Asia. :) Maybe in the future though.

    @Falzo said:

    probably have to deal with abuse a lot and it takes time to catch those. torrenters, plotters :'(

    While that is partly true, people who generally see extreme variability are the ones who run hundreds of benchmarks every month and get constantly throttled from time to time....or torrents. We've gotten very good with catching abuse nowadays that it is (almost) a non-issue.

    With nearly 5 PB of storage deployed, we have <5 users (same people) with complaints about performance issues on their storage servers since BF 2020. That is less than 5 users out of thousands.

    I think you have some storage servers from us so you probably know their performance yourself :)

    Thanked by 1miu
  • @hosthatch can you please take a look at my PM

  • FalzoFalzo Member

    @hosthatch said: I think you have some storage servers from us so you probably know their performance yourself

    yeah indeed, I am very happy and haven't seen any performance issue that I can remember. the above merely was a guess regarding the mentioned differences between nodes that some users seem to experience at different time.

    as you said could well be due to getting throttle as well. if there is some automation in place that throttles on heavy use I am totally fine with that. keeping the balance pro actively can only be appreciated ;-)

    Thanked by 1miu
  • Daniel15Daniel15 Veteran
    edited May 2021

    @hosthatch said: people who generally see extreme variability are the ones who run hundreds of benchmarks every month and get constantly throttled from time to time

    Is there a way to tell if I'm getting throttled? My storage server is idle most of the time, and I'm only running benchmarks to check the performance when it feels slow to me, but maybe that's enough to get throttled. My guess is that I'm one of the people with complaints about performance issues. I still feel like there's some sort of difference between the storage servers using E5-2680 processors (at least in Los Angeles) vs the ones using E5-2690 processors as I've only seen/experienced perf issues with the former. One of the reasons I got the storage VPS was to store all my photos on it (eg using PhotoStructure) but perhaps it's not optimized for a large number of small-ish (<10MB) files.

  • @Daniel15 said:

    @hosthatch said: people who generally see extreme variability are the ones who run hundreds of benchmarks every month and get constantly throttled from time to time

    Is there a way to tell if I'm getting throttled? My storage server is idle most of the time, and I'm only running benchmarks to check the performance when it feels slow to me, but maybe that's enough to get throttled. My guess is that I'm one of the people with complaints about performance issues. I still feel like there's some sort of difference between the storage servers using E5-2680 processors (at least in Los Angeles) vs the ones using E5-2690 processors as I've only seen/experienced perf issues with the former. One of the reasons I got the storage VPS was to store all my photos on it (eg using PhotoStructure) but perhaps it's not optimized for a large number of small-ish (<10MB) files.

    Yes, a notice in the panel by e.g. adding a batch saying the service is throttled could help in terms of transparency.
    Guess the difference in performance between between E5-2680v2 and E5-2690v2 is in general, not only on their storage servers. In their BF thread I've seen benchmarks that run on an even older CPU.

  • EmilEmil Member, Host Rep

    @Daniel15 said:

    @hosthatch said: people who generally see extreme variability are the ones who run hundreds of benchmarks every month and get constantly throttled from time to time

    Is there a way to tell if I'm getting throttled? My storage server is idle most of the time, and I'm only running benchmarks to check the performance when it feels slow to me, but maybe that's enough to get throttled. My guess is that I'm one of the people with complaints about performance issues. I still feel like there's some sort of difference between the storage servers using E5-2680 processors (at least in Los Angeles) vs the ones using E5-2690 processors as I've only seen/experienced perf issues with the former. One of the reasons I got the storage VPS was to store all my photos on it (eg using PhotoStructure) but perhaps it's not optimized for a large number of small-ish (<10MB) files.

    Currently there is no way of knowing, but notifications about this, null routes etc are planned for the upcoming refresh (actually a total rewrite) of our panel. I can't share exact details of thresholds and criterias etc for obvious reasons, but I am sure you don't need to worry about it. It's there to restrict abusers and malware infected VMs, not legitimate users with spiky usage.

    @webcraft said:

    @Daniel15 said:

    @hosthatch said: people who generally see extreme variability are the ones who run hundreds of benchmarks every month and get constantly throttled from time to time

    Is there a way to tell if I'm getting throttled? My storage server is idle most of the time, and I'm only running benchmarks to check the performance when it feels slow to me, but maybe that's enough to get throttled. My guess is that I'm one of the people with complaints about performance issues. I still feel like there's some sort of difference between the storage servers using E5-2680 processors (at least in Los Angeles) vs the ones using E5-2690 processors as I've only seen/experienced perf issues with the former. One of the reasons I got the storage VPS was to store all my photos on it (eg using PhotoStructure) but perhaps it's not optimized for a large number of small-ish (<10MB) files.

    Yes, a notice in the panel by e.g. adding a batch saying the service is throttled could help in terms of transparency.
    Guess the difference in performance between between E5-2680v2 and E5-2690v2 is in general, not only on their storage servers. In their BF thread I've seen benchmarks that run on an even older CPU.

    We have been working on phasing out E5-2690 (v1) CPUs for a while now, and it's almost finished. Only a few nodes left to go.

    Thanked by 1webcraft
  • Daniel15Daniel15 Veteran
    edited May 2021

    @webcraft said: Guess the difference in performance between between E5-2680v2 and E5-2690v2 is in general, not only on their storage servers

    The CPUs are very similar so I don't think it's directly related. My guess was that the 2680v2 nodes are older and have some other difference in the config (eg slower/older disks, different RAID card with different performance characteristics, not as much SSD caching, etc) that makes the disk IO slower compared to the 2690v2 nodes. HostHatch hasn't confirmed or denied this; they've just said that there's always going to be some variance between nodes. That's understandable at their sale price point and I completely understand, I just would have liked to see the same amazing performance on my 10TB storage VPS as other customers are seeing, that's all. I did have better perf on a Chicago storage VPS but I ended up transferring it to someone else since I didn't have a need for two different storage VPSes.

    @Emil said: but I am sure you don't need to worry about it. It's there to restrict abusers and malware infected VMs, not legitimate users with spiky usage.

    Thanks for the info :)

    Thanked by 1webcraft
  • pierrepierre Member
    edited May 2021

    @tetech said:
    Huh? They support IPv6 already.

    @Daniel15 said:
    All their locations support IPv6, and I think it's a /64 range in most locations. You just need to open a ticket to request that it be enabled.

    Ah! I didn't know that, my apologies.

  • EmilEmil Member, Host Rep

    @Daniel15 said:

    @webcraft said: Guess the difference in performance between between E5-2680v2 and E5-2690v2 is in general, not only on their storage servers

    The CPUs are very similar so I don't think it's directly related. My guess was that the 2680v2 nodes are older and have some other difference in the config (eg slower/older disks, different RAID card with different performance characteristics, not as much SSD caching, etc) that makes the disk IO slower compared to the 2690v2 nodes. HostHatch hasn't confirmed or denied this; they've just said that there's always going to be some variance between nodes. That's understandable at their sale price point and I completely understand, I just would have liked to see the same amazing performance on my 10TB storage VPS as other customers are seeing, that's all. I did have better perf on a Chicago storage VPS but I ended up transferring it to someone else since I didn't have a need for two different storage VPSes.

    @Emil said: but I am sure you don't need to worry about it. It's there to restrict abusers and malware infected VMs, not legitimate users with spiky usage.

    Thanks for the info :)

    It's true our nodes doesn't have identical hardware specs. Whenever we get a new batch of hardware we look into many aspects of how we can make our services better and offer you even more performance for the money than we already do. We pass on any efficiency savings to our customers. If we only keep getting the same configurations all the time then you would not see these improvements. Also, it wouldn't make sense to decommission hardware just for the sake of keeping stuff identical, if we did then pricing would look very different indeed.

    As mentioned earlier though we are swapping out some older NVMe nodes so these should be pretty consistent across the board. Storage nodes however is not as easy to replace, and keep in mind they are mainly designed for cost-effective archiving/backups and not IO intensive workloads. However you will see in most benchmarks that the performance is surprisingly good, so consider that a nice bonus instead of an expectation :)

    Thanked by 2fazar dosai
  • Daniel15Daniel15 Veteran

    @Emil said: will see in most benchmarks that the performance is surprisingly good, so consider that a nice bonus instead of an expectation

    Honestly I would pay 2 or 3 times what I'm paying now (or accept an equivalent reduction in the amount of space I've got) just to get speeds anywhere near what I see in some other benchmarks.

Sign In or Register to comment.