Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Advertise on LowEndTalk.com
Checkout time hdd vs nvme
New on LowEndTalk? Please read our 'Community Rules' by clicking on it in the right menu!

Checkout time hdd vs nvme

How important is hdd speed doing a svn checkout?

In a repository with lots of small files:
-* 15 seg : notebook with nvme (i5 8350u)
-* 75 seg : workstation with normal hdd (i5 6500)

¿That difference could be only thanks to nvme disk?

Comments

  • IonSwitchIonSwitch Member, Provider
    edited February 12

    Could that difference could be only thanks to nvme disk

    Short Answer

    Absolutely.

    Long Answer

    When you write a file to disk, the disk has to find a location on disk and write to it. On a spinny disk the spindle is flying around and each individual file is being put at a different location on disk. The small writes/files and high volume of them make it difficult for the disk to avoid constant seeking between files, and you get what almost looks like a random IO pattern. Random IO is really bad for spinny disks.

    On an SSD or NVME, the data is cached instantly into memory on the SSD with nearly no latency. Theres no fraction of a second for the head to seek on the disk. Modern SSD or NVME have buffers that begin to approach memory speed.

    If you consider saving a file an IO operation (in practice, its more than one, but lets just call it one), then a spinny disk (5400 rpm or 7200 rpm) is performing 80-120 IO operations (IOPS) per second. Lets assume that actually means 80-120 files, it doesnt, but lets just use it for a comparison.

    Now your NVME is able to do somewhere between 50k and 500k IOPS (depending on model). 50,000-500,000 files per second (again, just for comparison sake).

    The 80-120 vs 50,000-500,000 IOPS is what you are seeing play out.

    Thanked by 2ese_enzo vpsGOD

    IonSwitch, LLC | VPS, Colocation, and Dedicated Servers | Seattle, WA
    | AS16584 | Looking Glass | Email me! [email protected]

  • @IonSwitch said:

    Could that difference could be only thanks to nvme disk

    Short Answer

    Absolutely.

    Long Answer

    When you write a file to disk, the disk has to find a location on disk and write to it. On a spinny disk the spindle is flying around and each individual file is being put at a different location on disk. The small writes/files and high volume of them make it difficult for the disk to avoid constant seeking between files, and you get what almost looks like a random IO pattern. Random IO is really bad for spinny disks.

    On an SSD or NVME, the data is cached instantly into memory on the SSD with nearly no latency. Theres no fraction of a second for the head to seek on the disk. Modern SSD or NVME have buffers that begin to approach memory speed.

    If you consider saving a file an IO operation (in practice, its more than one, but lets just call it one), then a spinny disk (5400 rpm or 7200 rpm) is performing 80-120 IO operations (IOPS) per second. Lets assume that actually means 80-120 files, it doesnt, but lets just use it for a comparison.

    Now your NVME is able to do somewhere between 50k and 500k IOPS (depending on model). 50,000-500,000 files per second (again, just for comparison sake).

    The 80-120 vs 50,000-500,000 IOPS is what you are seeing play out.

    Thanks, that is an answer :smile:

  • marvelmarvel Member without signature

    @IonSwitch said:

    Could that difference could be only thanks to nvme disk

    Short Answer

    Absolutely.

    Long Answer

    When you write a file to disk, the disk has to find a location on disk and write to it. On a spinny disk the spindle is flying around and each individual file is being put at a different location on disk. The small writes/files and high volume of them make it difficult for the disk to avoid constant seeking between files, and you get what almost looks like a random IO pattern. Random IO is really bad for spinny disks.

    On an SSD or NVME, the data is cached instantly into memory on the SSD with nearly no latency. Theres no fraction of a second for the head to seek on the disk. Modern SSD or NVME have buffers that begin to approach memory speed.

    If you consider saving a file an IO operation (in practice, its more than one, but lets just call it one), then a spinny disk (5400 rpm or 7200 rpm) is performing 80-120 IO operations (IOPS) per second. Lets assume that actually means 80-120 files, it doesnt, but lets just use it for a comparison.

    Now your NVME is able to do somewhere between 50k and 500k IOPS (depending on model). 50,000-500,000 files per second (again, just for comparison sake).

    The 80-120 vs 50,000-500,000 IOPS is what you are seeing play out.

    This is only the case with synchronous data writes where the application waits until it gets confirmation that the data has been REALLY written to the disk. e.g. a database server where data consistency is very important and you don't want to lose any data on a power outage or kernel crash.

    With asynchronous writes, which are most writes like file copying and SVN/GIT checkouts the data is written to RAM first and the app just assumes it's written to disk when that part is finished. Next the RAM is flushed to disk. So async writes are a whole LOT quicker than sync writes.

    ZFS solves these issues partly btw by adding a SSD/NVMe SLOG device to the slow spinning disk pool, where the synchronous data is written to first, then later it's flushed to disk. This prevents your disk perfomance being killed with small sync writes across your entire disk (or multiple disks) like you already mentioned.

    So if you have enough RAM and SVN checkouts are asynchronous (which is the part I'm not sure about) you should not notice any perfomance difference between SSD and HDD when doing a checkout. If it's too big to fit all in RAM and it has to wait for the disk to flush the data to then yes, your perfomance will crawl.

  • i'm nyimak

    kuat dilakoni, ra kuat tinggal ngopi

Sign In or Register to comment.