[SPDK] Re: Rocksdb over SPDK/Blobfs

Hi Dimitra,

I think what's happening here is that your database is small enough that it fits mostly (or even entirely) in the kernel page cache.  This means that when using the kernel NVMe driver + filesystem, you are avoiding a lot of reads to the SSD since the data is cached in memory.

This SPDK GitHub issue describes this phenomenon a bit more:  https://github.com/spdk/spdk/issues/1555

I would be interested to see the comparison if you make your database size much bigger, such that it cannot all fit into page cache.

Thanks,

Jim


On 11/25/20, 6:30 AM, "Dimitra Giantsidi" <dimitra.giantsidi(a)gmail.com> wrote:

    Hi everyone,

    I am running the rocksdb::RandomTransaction benchmark with *1 thread* with
    the following specifications:

    RocksDB:    version 5.14
    Date:       Wed Nov 25 13:05:48 2020
    CPU:        16 * Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz
    CPUCache:   16384 KB
    Keys:       16 bytes each
    Values:     1024 bytes each (1024 bytes after compression)
    Entries:    5000000
    Prefix:    0 bytes
    Keys per prefix:    0
    RawSize:    4959.1 MB (estimated)
    FileSize:   4959.1 MB (estimated)
    Write rate: 0 bytes/second
    Read rate: 0 ops/second
    Compression: NoCompression
    Memtablerep: skip_list
    Perf Level: 1
    -----------------------------------------------------------------------------------------------------------------------

    When using the kernel NVMe driver I have better performance (x 5 times). In
    particular the db_bench output is:
    NVMe driver: randomtransaction :      *16.727 micros/op 59782 ops/sec;*
     1.5 MB/s (transactions:5000000 aborts:0)

    SPDK: randomtransaction :      *71.226 micros/op 14039 ops/sec;*    0.3
    MB/s (transactions:5000000 aborts:0)

    I am using SPDK v18.01.1 and DPDK v18.02. The SPDK cache is 4GB and
    apparently I cannot increase it in my platform.

    I am not sure how to explain this behavior. I would appreciate any
    help/comment on the result.

    Thanks,
    Dimitra

    -- 
    Giantsidi Dimitra

    e-mail:dimitra.giantsidi(a)gmail.com
    _______________________________________________
    SPDK mailing list -- spdk(a)lists.01.org
    To unsubscribe send an email to spdk-leave(a)lists.01.org