Re: Performance Difference between files getting opened with IoContext.RANDOM vs IoContext.READ

Hi,

this seems to be aspecial case in FlatVectors, because normally theres a 
separate method to open an IndexInput for checksumming:

https://github.com/apache/lucene/blob/524ea208c870861a719f21b1ea48943c8b7520da/lucene/core/src/java/org/apache/lucene/store/Directory.java#L155-L157

Could you open an issue, it looks like it is not always used? I know 
there are some problems if the IndexInput is used for multiple things 
like reading, merging and/or checksumming at same time. Some code tries 
to reuse already opened index inputs also for merging. But for this case 
I think it might be better to open a separate IndexInput and not clone 
an existing one for checksumming?

The first link should of course open the IndexInput with RANDOM, because 
during normal reading of vectors this is a must. Generally although the 
checksumming is slower it should not be a big issue, because it won't 
affect searches, only merging of segments. And there the throughput 
should be high, but not top priority.

Uwe

Am 01.10.2024 um 04:52 schrieb Navneet Verma:
> Hi Uwe and Mike,
> Thanks for providing such a quick response. Let me try to ans few things
> here:
>
>
>
>
>
> *In addition, inLucene 9.12 (latest 9.x) version released today there are
> some changesto ensure that checksumming is always done with
> IOContext.READ_ONCE(which uses READ behind the scenes).*
> I didn't find any such change for FlatVectorReaders
> <https://github.com/apache/lucene/blob/branch_9_12/lucene/core/src/java/org/apache/lucene/codecs/lucene99/Lucene99FlatVectorsReader.java#L68-L77>,
> even though I checked the BufferedChecksumInput
> <https://github.com/apache/lucene/blob/branch_9_12/lucene/core/src/java/org/apache/lucene/store/BufferedChecksumIndexInput.java#L31-L35>
> and CheckedSumInput
> <https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/store/ChecksumIndexInput.java#L25>,
> CodecUtil
> <https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/codecs/CodecUtil.java#L606-L621>
> in 9.12 version. Please point me to the right file if I am missing
> something here. I can see the same for lucene version 10
> <https://github.com/apache/lucene/blob/branch_10_0/lucene/core/src/java/org/apache/lucene/codecs/lucene99/Lucene99FlatVectorsReader.java#L69-L77>
> too.
>
> Mike on the question of what is RANDOM vs READ context doing we found this
> information related to MADV online.
>
> MADV_RANDOM Expect page references in random order. (Hence, read ahead may
> be less useful than normally.)
> MADV_SEQUENTIAL Expect page references in sequential order. (Hence, pages
> in the given range can be aggressively read ahead, and may be freed soon
> after they are accessed.)
> MADV_WILLNEED Expect access in the near future. (Hence, it might be a good
> idea to read some pages ahead.)
>
> This tells me that MADV_RANDOM random for checksum is not good as it will
> consume more read cycles given the sequential nature of the checksum.
>
>
>
>
>
>
>
> *One simple workaround an application can do is to ask MMapDirectory
> topre-touch all bytes/pages in .vec/.veq files -- this asks the OS to
> cacheall of those bytes into page cache (if there is enough free RAM).  We
> dothis at Amazon (product search) for our production searching
> processes.Otherwise paging in all .vec/.veq pages via random access
> provoked throughHNSW graph searching is crazy slow...*
> Did you mean the preload functionality offered by MMapDirectory here? I can
> try this to see if that helps. But I doubt that in this case.
>
> On opening the issue, I am working through some reproducible benchmarks
> before creating a gh issue. If you believe I should create a GH issue first
> I can do that. As it might take me sometime to build reproducible
> benchmarks.
>
> Thanks
> Navneet
>
>
> On Mon, Sep 30, 2024 at 3:08 AM Uwe Schindler <[email protected]> wrote:
>> Hi,
>>
>> please also note: In Lucene 10 there checksum IndexInput will always be
>> opened with IOContext.READ_ONCE.
>>
>> If you want to sequentially read a whole index file for other reason
>> than checksumming, please pass the correct IOContext. In addition, in
>> Lucene 9.12 (latest 9.x) version released today there are some changes
>> to ensure that checksumming is always done with IOContext.READ_ONCE
>> (which uses READ behind scenes).
>>
>> Uwe
>>
>> Am 29.09.2024 um 17:09 schrieb Michael McCandless:
>>> Hi Navneet,
>>>
>>> With RANDOM IOcontext, on modern OS's / Java versions, Lucene will hint
> the
>>> memory mapped segment that the IO will be random using madvise POSIX API
>>> with MADV_RANDOM flag.
>>>
>>> For READ IOContext, Lucene maybe hits with MADV_SEQUENTIAL, I'm not
> sure.
>>> Or maybe it doesn't hint anything?
>>>
>>> It's up to the OS to then take these hints and do something
> "interesting"
>>> to try to optimize IO and page caching based on these hints.  I think
>>> modern Linux OSs will readahead (and pre-warm page cache) for
>>> MADV_SEQUENTIAL?  And maybe skip page cache and readhead for
> MADV_RANDOM?
>>> Not certain...
>>>
>>> For computing checksum, which is always a sequential operation, if we
> use
>>> MADV_RANDOM (which is stupid), that is indeed expected to perform worse
>>> since there is no readahead pre-caching.  50% worse (what you are
> seeing)
>>> is indeed quite an impact ...
>>>
>>> Maybe open an issue?  At least for checksumming we should open even .vec
>>> files for sequential read?  But, then, if it's the same IndexInput which
>>> will then be used "normally" (e.g. for merging), we would want THAT one
> to
>>> be open for random access ... might be tricky to fix.
>>>
>>> One simple workaround an application can do is to ask MMapDirectory to
>>> pre-touch all bytes/pages in .vec/.veq files -- this asks the OS to
> cache
>>> all of those bytes into page cache (if there is enough free RAM).  We do
>>> this at Amazon (product search) for our production searching processes.
>>> Otherwise paging in all .vec/.veq pages via random access provoked
> through
>>> HNSW graph searching is crazy slow...
>>>
>>> Mike McCandless
>>>
>>> http://blog.mikemccandless.com
>>>
>>> On Sun, Sep 29, 2024 at 4:06 AM Navneet Verma <[email protected]
>>> wrote:
>>>
>>>> Hi Lucene Experts,
>>>> I wanted to understand the performance difference between opening and
>>>> reading the whole file using an IndexInput with IoContext as RANDOM vs
>>>> READ.
>>>>
>>>> I can see .vec files(storing the flat vectors) are opened with RANDOM
> and
>>>> whereas dvd files are opened as READ. As per my testing with files
> close to
>>>> size 5GB storing (~1.6M docs with each doc 3072 bytes), I can see that
> when
>>>> full file checksum validation is happening for a file opened via READ
>>>> context it is faster than RANDOM. The amount of time difference I am
> seeing
>>>> is close to 50%. Hence the performance question is coming up, I wanted
> to
>>>> understand is this understanding correct?
>>>>
>>>> Thanks
>>>> Navneet
>>>>
>> --
>> Uwe Schindler
>> Achterdiek 19, D-28357 Bremen
>> https://www.thetaphi.de
>> eMail: [email protected]
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [email protected]
>> For additional commands, e-mail: [email protected]
>>
-- 
Uwe Schindler
Achterdiek 19, D-28357 Bremen
https://www.thetaphi.de
eMail: [email protected]