[SPDK] Re: Rocksdb over SPDK/Blobfs

Okay, thanks a lot!

Dimitra

On Wed, 25 Nov 2020 at 18:01, Harris, James R <james.r.harris(a)intel.com>
wrote:

> The SPDK cache is only used for write buffering and readahead (for
> compaction).  So if your workload is only doing random GET operations, the
> SPDK cache will have little to no effect.
>
> -Jim
>
>
> On 11/25/20, 8:39 AM, "Dimitra Giantsidi" <dimitra.giantsidi(a)gmail.com>
> wrote:
>
>     Hi Jim,
>
>     Thanks for your prompt reply! That was really fast!
>     Hmm.. since the kernel page cache has no limit, that makes sense.
>     However, my workload size is estimated to be 4959.1 MB, namely, almost
> as
>     much as the SPDK cache. I would assume that in this setting SPDK cache
>     would host most of the data, so both versions should be equivalent
> (more or
>     less), no?
>
>     Assuming that SPDK cache can be as much as the system's memory (or at
> least
>     bigger than the DB size) -- should I expect that SPDK performs better
> than
>     NVMe driver?
>
>     (I am running some experiments with the configurations you proposed
> and I
>     will post them as soon as possible :) )
>
>
>     Thank you for your help,
>     Dimitra
>
>     On Wed, 25 Nov 2020 at 17:07, Harris, James R <
> james.r.harris(a)intel.com>
>     wrote:
>
>     > Hi Dimitra,
>     >
>     > I think what's happening here is that your database is small enough
> that
>     > it fits mostly (or even entirely) in the kernel page cache.  This
> means
>     > that when using the kernel NVMe driver + filesystem, you are
> avoiding a lot
>     > of reads to the SSD since the data is cached in memory.
>     >
>     > This SPDK GitHub issue describes this phenomenon a bit more:
>     > https://github.com/spdk/spdk/issues/1555
>     >
>     > I would be interested to see the comparison if you make your
> database size
>     > much bigger, such that it cannot all fit into page cache.
>     >
>     > Thanks,
>     >
>     > Jim
>     >
>     >
>     > On 11/25/20, 6:30 AM, "Dimitra Giantsidi" <
> dimitra.giantsidi(a)gmail.com>
>     > wrote:
>     >
>     >     Hi everyone,
>     >
>     >     I am running the rocksdb::RandomTransaction benchmark with *1
> thread*
>     > with
>     >     the following specifications:
>     >
>     >     RocksDB:    version 5.14
>     >     Date:       Wed Nov 25 13:05:48 2020
>     >     CPU:        16 * Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz
>     >     CPUCache:   16384 KB
>     >     Keys:       16 bytes each
>     >     Values:     1024 bytes each (1024 bytes after compression)
>     >     Entries:    5000000
>     >     Prefix:    0 bytes
>     >     Keys per prefix:    0
>     >     RawSize:    4959.1 MB (estimated)
>     >     FileSize:   4959.1 MB (estimated)
>     >     Write rate: 0 bytes/second
>     >     Read rate: 0 ops/second
>     >     Compression: NoCompression
>     >     Memtablerep: skip_list
>     >     Perf Level: 1
>     >
>     >
> -----------------------------------------------------------------------------------------------------------------------
>     >
>     >     When using the kernel NVMe driver I have better performance (x 5
>     > times). In
>     >     particular the db_bench output is:
>     >     NVMe driver: randomtransaction :      *16.727 micros/op 59782
> ops/sec;*
>     >      1.5 MB/s (transactions:5000000 aborts:0)
>     >
>     >     SPDK: randomtransaction :      *71.226 micros/op 14039
> ops/sec;*    0.3
>     >     MB/s (transactions:5000000 aborts:0)
>     >
>     >     I am using SPDK v18.01.1 and DPDK v18.02. The SPDK cache is 4GB
> and
>     >     apparently I cannot increase it in my platform.
>     >
>     >     I am not sure how to explain this behavior. I would appreciate
> any
>     >     help/comment on the result.
>     >
>     >     Thanks,
>     >     Dimitra
>     >
>     >     --
>     >     Giantsidi Dimitra
>     >
>     >     e-mail:dimitra.giantsidi(a)gmail.com
>     >     _______________________________________________
>     >     SPDK mailing list -- spdk(a)lists.01.org
>     >     To unsubscribe send an email to spdk-leave(a)lists.01.org
>     >
>     > _______________________________________________
>     > SPDK mailing list -- spdk(a)lists.01.org
>     > To unsubscribe send an email to spdk-leave(a)lists.01.org
>     >
>
>
>     --
>     Giantsidi Dimitra
>
>     e-mail:dimitra.giantsidi(a)gmail.com
>     _______________________________________________
>     SPDK mailing list -- spdk(a)lists.01.org
>     To unsubscribe send an email to spdk-leave(a)lists.01.org
>
> _______________________________________________
> SPDK mailing list -- spdk(a)lists.01.org
> To unsubscribe send an email to spdk-leave(a)lists.01.org
>


-- 
Giantsidi Dimitra

e-mail:dimitra.giantsidi(a)gmail.com