Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Performance Issue - 2 x SSD Evo 840 Pro 1TB (RAID1) with Adaptec 6405E Hardware Raid Controller
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Performance Issue - 2 x SSD Evo 840 Pro 1TB (RAID1) with Adaptec 6405E Hardware Raid Controller

Hi, we have a dedi with 2 disks SSD Evo 840 Pro 1TB connected to an Adaptec 6405E in a RAID1 and we get this results with dd:

[root@hw3 adaptec]# dd if=/dev/zero of=/home/output.img bs=8k count=256k
262144+0 records in
262144+0 records out
2147483648 bytes (2.1 GB) copied, 12.2099 s, 176 MB/s
[root@hw3 adaptec]# dd if=/dev/zero of=/home/output.img bs=8k count=256k
262144+0 records in
262144+0 records out
2147483648 bytes (2.1 GB) copied, 11.6456 s, 184 MB/s
[root@hw3 adaptec]# dd if=/dev/zero of=/home/output.img bs=8k count=256k
262144+0 records in
262144+0 records out
2147483648 bytes (2.1 GB) copied, 33.5191 s, 64.1 MB/s
[root@hw3 adaptec]# dd if=/dev/zero of=/home/output.img bs=8k count=256k
262144+0 records in
262144+0 records out
2147483648 bytes (2.1 GB) copied, 50.2451 s, 42.7 MB/s

Anyone can tell us what may be wrong in here ?

We should get about 1.5GB/s or more, right ?

Thanks

Comments

  • i had a similar problem in the past with 850 evo , i just change a lsi controller and was all ok. Maybe is something with this controler... at least in my case the hdd was new.

    Thanked by 1KeyJey
  • launchvpslaunchvps Member, Patron Provider

    Your speeds may have to do with caching...

    Does the RAID card have a cache and battery? If so, be sure this is enabled and that "write back" is enabled. Then perform your tests.

    You may be also able to force write back on without a battery unit, but this should only be enabled for testing. You can lose data if this is enabled in production.

    I don't have much experience with Adaptec cards, but the LSI (Now Broadcom?) cards we use have some additional performance settings when a BBU unit is used with SSDs.

    So, I recommend playing with the caching and write back settings and re-run tests to see where you stand. That may give you a jumping off point to understand where your speeds are falling off.

    Finally, a RAID 1 may not get you the 1.5GB/s range. You may need a larger RAID set to get there, but you should be seeing higher speeds that you are currently seeing.

    Thanked by 1KeyJey
  • MikePTMikePT Moderator, Patron Provider, Veteran

    There's something wrong there, though dd isn't an accurate test. Still, those results are pretty bad.

    Is it connected with 6Gbps, or 15Gbps?

    Have you tried another RAID Controller?
    Sometimes, software RAID is faster than hardware RAID, due to the controller and port speeds.

  • See this:

  • MikePTMikePT Moderator, Patron Provider, Veteran

    @KeyJey said:
    See this:

    Bad HW Raid card, swap it for another one.

  • PioHostPioHost Member
    edited February 2017

    if you want a true test then run the following

    hdparm -Tt /dev/sda
    hdparm -Tt /dev/sdb

    i have the same results from my dedicated server with similar 1tb drives

    https://forum.piohost.co.uk/showthread.php?tid=3

  • Try it without the RAID card - hardware RAID is mostly pointless when you are using SSDs. Its more likely to hurt than help.

  • @Microlinux said:
    Try it without the RAID card - hardware RAID is mostly pointless when you are using SSDs. Its more likely to hurt than help.

    that depends on the raid controller. If you're using a 10 years old model (for example adaptec 5405) with ssds it's clear that the controller will slow down the performance. But current models are fine for ssd usage. We're using adaptec series 6 models with ssds without problems. Series 7 or 8 would be even better.

  • @rushway said:

    @Microlinux said:
    Try it without the RAID card - hardware RAID is mostly pointless when you are using SSDs. Its more likely to hurt than help.

    that depends on the raid controller. If you're using a 10 years old model (for example adaptec 5405) with ssds it's clear that the controller will slow down the performance. But current models are fine for ssd usage. We're using adaptec series 6 models with ssds without problems. Series 7 or 8 would be even better.

    You can use whatever model you want, but save for a very small number of scenarios, there isn't really any real world benefit to using hardware RAID with SSDs.

  • ihadpihadp Member
    edited February 2017

    @Microlinux said:

    @rushway said:

    @Microlinux said:
    Try it without the RAID card - hardware RAID is mostly pointless when you are using SSDs. Its more likely to hurt than help.

    that depends on the raid controller. If you're using a 10 years old model (for example adaptec 5405) with ssds it's clear that the controller will slow down the performance. But current models are fine for ssd usage. We're using adaptec series 6 models with ssds without problems. Series 7 or 8 would be even better.

    You can use whatever model you want, but save for a very small number of scenarios, there isn't really any real world benefit to using hardware RAID with SSDs.

    You also loose TRIM support, which is rather important especially for EVO/PRO/etc. drives and good to have with enterprise drives. Garbage collection without TRIM can really cripple the array performance as time progresses.

    MDADM/ZFS (software RAID) supports TRIM out of the box.

    HBA + MDADM/ZFS = Good to Go!

    Thanked by 1KeyJey
  • mikhomikho Member, Host Rep

    If you want to run ESXi and the likes, you are stuck with hardware raid since it doesn't support software (last I checked)

    Thanked by 1Clouvider
  • PieHasBeenEatenPieHasBeenEaten Member, Host Rep
    edited February 2017

    Yeah running hw raid is a total waste with ssds. We had a bigger failure rate with the ssds in hw raid than sw raid setups. Also are the firmwires up to date also on the ssds.

    Thanked by 1KeyJey
  • Here is the info from the controller:

    [root@hw3 adaptec]# arcconf GETCONFIG 1
    Controllers found: 1
    ----------------------------------------------------------------------
    Controller information
    ----------------------------------------------------------------------
       Controller Status                        : Optimal
       Channel description                      : SAS/SATA
       Controller Model                         : Adaptec 6405E
       Controller Serial Number                 : 2A421215CCA
       Controller World Wide Name               : 50000D1105D33E00
       Controller Alarm                         : Disabled
       Physical Slot                            : 4
       Temperature                              : 61 C/ 141 F (Normal)
       Installed memory                         : 128 MB
       Global task priority                     : High
       Performance Mode                         : Big Block Bypass
       Host bus type                            : PCIe
       Host bus speed                           : 5000 MHz
       Host bus link width                      : 1 bit(s)/link(s)
       PCI Device ID                            : 651
       Stayawake period                         : Disabled
       Spinup limit internal drives             : 0
       Spinup limit external drives             : 0
       Defunct disk drive count                 : 0
       Logical devices/Failed/Degraded          : 1/0/0
       NCQ status                               : Enabled
       Statistics data collection mode          : Enabled
       --------------------------------------------------------
       RAID Properties
       --------------------------------------------------------
       Copyback                                 : Disabled
       Automatic Failover                       : Enabled
       Background consistency check             : Disabled
       Background consistency check period      : 30
       --------------------------------------------------------
       Controller Version Information
       --------------------------------------------------------
       BIOS                                     : 5.3-0 (19198)
       Firmware                                 : 5.3-0 (19198)
       Driver                                   : 1.2-1 (41066)
       Boot Flash                               : 0.0-0 (19198)
       SEEPROM (Load version/ Flash version)    : 2/ 8
    
    ----------------------------------------------------------------------
    Logical device information
    ----------------------------------------------------------------------
    Logical Device number 0
       Logical Device name                      : RAID1
       Block Size of member drives              : 512 Bytes
       RAID level                               : 1
       Unique Identifier                        : F95303A9
       Status of Logical Device                 : Optimal
       Additional details                       : Quick initialized
       Size                                     : 953334 MB
       Parity space                             : 953344 MB
       Interface Type                           : Serial ATA
       Device Type                              : HDD
       Read-cache setting                       : Disabled
       Read-cache status                        : Off
       Write-cache setting                      : Disabled
       Write-cache status                       : Off
       Partitioned                              : Yes
       Protected by Hot-Spare                   : No
       Bootable                                 : Yes
       Failed stripes                           : No
       Power settings                           : Disabled
       --------------------------------------------------------
       Logical Device segment information
       --------------------------------------------------------
       Segment 0                                : Present (953869MB, SATA, SSD, Connector:0, Device:0) S1D9NSAF728578P
       Segment 1                                : Present (953869MB, SATA, SSD, Connector:0, Device:1) S1D9NSAF728659J
    
    
    ----------------------------------------------------------------------
    Physical Device information
    ----------------------------------------------------------------------
          Device #0
             Device is a Hard drive
             State                              : Online
             Block Size                         : 512 Bytes
             Supported                          : Yes
             Transfer Speed                     : SATA 6.0 Gb/s
             Reported Channel,Device(T:L)       : 0,0(0:0)
             Reported Location                  : Connector 0, Device 0
             Vendor                             : ATA
             Model                              : Samsung SSD 840
             Firmware                           : EXT0BB6Q
             Serial number                      : S1D9NSAF728578P
             World-wide name                    : 50025388A059B3A5
             Reserved Size                      : 538264 KB
             Used Size                          : 953344 MB
             Unused Size                        : 64 KB
             Total Size                         : 953869 MB
             Write Cache                        : Enabled (write-back)
             FRU                                : None
             S.M.A.R.T.                         : No
             S.M.A.R.T. warnings                : 0
             Power State                        : Full rpm
             Supported Power States             : Full power,Powered off
             SSD                                : Yes
             Temperature                        : Not Supported
             NCQ status                         : Enabled
          ----------------------------------------------------------------
          Device Phy Information
          ----------------------------------------------------------------
             Phy #0
                PHY Identifier                  : 0
                SAS Address                     : 3000000000000000
                Attached PHY Identifier         : 0
                Attached SAS Address            : 50000D1105D33E00
    
          Device #1
             Device is a Hard drive
             State                              : Online
             Block Size                         : 512 Bytes
             Supported                          : Yes
             Transfer Speed                     : SATA 6.0 Gb/s
             Reported Channel,Device(T:L)       : 0,1(1:0)
             Reported Location                  : Connector 0, Device 1
             Vendor                             : ATA
             Model                              : Samsung SSD 840
             Firmware                           : EXT0BB6Q
             Serial number                      : S1D9NSAF728659J
             World-wide name                    : 50025388A059B3F6
             Reserved Size                      : 538264 KB
             Used Size                          : 953344 MB
             Unused Size                        : 64 KB
             Total Size                         : 953869 MB
             Write Cache                        : Enabled (write-back)
             FRU                                : None
             S.M.A.R.T.                         : No
             S.M.A.R.T. warnings                : 0
             Power State                        : Full rpm
             Supported Power States             : Full power,Powered off
             SSD                                : Yes
             Temperature                        : Not Supported
             NCQ status                         : Enabled
          ----------------------------------------------------------------
          Device Phy Information
          ----------------------------------------------------------------
             Phy #0
                PHY Identifier                  : 0
                SAS Address                     : 3000000000000001
                Attached PHY Identifier         : 1
                Attached SAS Address            : 50000D1105D33E00
    
    
    
    ----------------------------------------------------------------------
    Connector information
    ----------------------------------------------------------------------
    Connector #0
       Connector Name                           : CN0
    
       -------------------------------------------------------------------
       Lane Information
       -------------------------------------------------------------------
       Lane #0
          Channel ID                            : 0
          Device ID                             : 0
          SAS Address                           : 50000D1105D33E00
          PHY Identifier                        : 0
          ----------------------------------------------------------------
          Lane SAS Phy Information
          ----------------------------------------------------------------
             No SAS Phy information available for lane #0
    
       Lane #1
          Channel ID                            : 0
          Device ID                             : 1
          SAS Address                           : 50000D1105D33E00
          PHY Identifier                        : 1
          ----------------------------------------------------------------
          Lane SAS Phy Information
          ----------------------------------------------------------------
             No SAS Phy information available for lane #1
    
       Lane #2
          Channel ID                            : 0
          Device ID                             : 2
          SAS Address                           : 50000D1105D33E00
          PHY Identifier                        : 2
          ----------------------------------------------------------------
          Lane SAS Phy Information
          ----------------------------------------------------------------
             No SAS Phy information available for lane #2
    
       Lane #3
          Channel ID                            : 0
          Device ID                             : 3
          SAS Address                           : 50000D1105D33E00
          PHY Identifier                        : 3
          ----------------------------------------------------------------
          Lane SAS Phy Information
          ----------------------------------------------------------------
             No SAS Phy information available for lane #3
    
    
    
    Command completed successfully.
    [root@hw3 adaptec]#
    

    Yes, we will remove the RAID controller and that's it.

    Thanks for your comments, very helpful !

  • MikePT said: Is it connected with 6Gbps, or 15Gbps?

    As you could see, it's a 6GBps port :(

    Thanked by 1MikePT
  • ClouviderClouvider Member, Patron Provider
    edited February 2017

    @KeyJey said:

    MikePT said: Is it connected with 6Gbps, or 15Gbps?

    As you could see, it's a 6GBps port :(

    6GBps = 48 Gbps.

    Your expectations are unrealistic. 6Gbps port gives you only theoretically 0.75GBps so two ports theoretically 1.5 GBps, which doesn't mean that this is the data rate you will achieve. I'm yet to see a SATA III SSD combo in RAID 1 that is able to work with that data rate. If you do get similar results to 1.5GB/s that would be the data rate of your RAID Card's / RAM cache, and not the one of drives.

  • Obviously the data points suggest that the raid controller is one culprit, actually the major one.

    There are many misunderstandings and pitfalls, particularly when SSDs are involved.

    Nice example: What is Raid 1? It boils down to writing out two copies of the buffer to two disks. Which is something any halfway decent OS can do easily - and quickly. Now, let's look at the physical side: data goes from buffer through cpu bridge and then via some kind of serdes directly to sata PHY and then though the sata cables to the drives.

    Doing the same with hw raid controller simply pushes the mirroring to the raid card - which is guaranteed to be slower than the cpu bridge as there is no raid processing whatsoever involved, not even simple striping as in raid 0.

    In other words: Using a raid controller card in your situation is bound to slow things. Plus it is a wall for TRIM.

    To make things worse, your raid controller has a cache. Do not assume that disabling the cache on the raid card does actually disable it! In fact, it will almost certainly use it because that's it only way to cope with fast data input and because that cache is way bigger than its own RAM (which usually is ridiculously small but sufficient) and because that's how writing to disks works.

    So, using a raid 1 controller you simply add a) a slower cpu and b) yet another buffer into the chain.

    And that's what you see in your dd values.

Sign In or Register to comment.