Question about the ticket #743 omindex: delay libmagic checks

YuLun Cai <[email protected]> Fri, 21 Apr 2017 01:52:38 +0800
Newsgroups gmane.comp.search.xapian.devel
Message-ID <CAC2f5hvscs1N4fZGWF9=aadBVdX4fz1KUWQ7c_k9tjg4-h6GvQ@mail.gmail.com>
Hi,

I'm working on the ticket #743 omindex: delay libmagic checks
<https://trac.xapian.org/ticket/743>. As the ticket's
Description mention, the call to libmagic is expensive than call the stat,
so we can check the size by call the stat to get size before call
libmagic to get a mime type.

But how about the timestamps check? since timestamps check need to iterate
the DB to check if the file has been indexed and hasn't changed(in
`index_check_existing` function in omega\index_file.cc), so it is expensive
too. Should we call the libmagic before or after the timestamps, or do we
have another way to check the timestamps?

What's more, how should we write tests to prove the omindex works
correctly, to generate some practical directories and use omindex to index
it then check the things in DB?

Thanks.