Chat history progress - GSoC 2017 (was: Timeline for GSoC 2017 project (chat history improvements))

Paulo Lieuthier <[email protected]> Tue, 6 Jun 2017 11:05:26 -0300
Newsgroups	gmane.comp.kde.devel.kopete
Message-ID	<[email protected]>
Hi, Pali!

Really sorry for the delay! I will do my best to send my reports on the saturdays.

I'm keeping my work on Github [1]. I'm trying to keep my commits very organized,
so you can easily track my progress, and maybe cherry-pick bugfixes if they are
worth it.

That's what I did this week:

* I got the XML-based plugin to build and work on top of the kf5 branch. It took
some time for me to grok it, but now I'm more familiar with the codebase. There
is a bug when creating the history XML files, which I fixed in [2] (should I
send a patch to review board?).

* I have done some research about SQLite full-text search capabilities. I think
this it is a great option. Metadata is stored separated from the text indexes
and the indexes are automatically updated when some record changes [3]. Also,
there is built-in support for relevancy-based ordering and stemming [5].

We haven't dicussed much about which should be kept or removed from the current
plugins, but I don't think there is much to bikeshed here: I must aim for a
simple and fast storage with a complete schema that works for all protocols.

Speaking of schema, that's my main goal for this week. I believe it will be
similar to what will sent my previously, but not quite. I'm thinking of
something more general, that is not tied up to text or simple groups (multimedia
messages and channels must be first-class).

I'm looking forward to get feedback.

Paulo

[1] https://github.com/paulolieuthier/kopete/commits/history-plugin
[2]
https://github.com/paulolieuthier/kopete/commit/a901f0455f228a467dd9e5f8dd990535b9da5873
[3] https://sqlite.org/fts5.html#external_content_tables
[4] https://sqlite.org/fts5.html#_summary_of_technical_differences_
[5] https://sqlite.org/fts5.html#tokenizers

On 05-06-2017 16:05, Pali Rohár wrote:
> Hi Paulo!
> 
> Now one week passed and we would like to hear from you some status 
> update...
> 
> On Monday 29 May 2017 04:04:57 Paulo Lieuthier wrote:
>> May 29 – June 2
>> * Become familiar with Kopete’s current history plugins’ codebases.
>> * Discuss with ML the reasoning for deprecating previous plugins, and
>> also features that should be kept and new ones.
>> * Get the XML-based history plugin to build on top of kf5 branch.
>>
>> June 5 – June 9
>> * Wrap my around of a good and sufficient schema for the
>> conversations history. * See how other apps manage messages history.
>> * Search for a database solution, both schema-friendly and text
>> search-friendly (SQLite, Firebird, Xapian)?.
>
signature.asc (application/pgp-signature, 833 B)