Re: Timeline for GSoC 2017 project (chat history improvements)

On Wednesday 31 May 2017 11:20:41 Paulo Lieuthier wrote:
> On 29-05-2017 04:41, Pali Rohár wrote:
> > On Sunday 28 May 2017 23:04:57 Paulo Lieuthier wrote:
> >> Hey, Pali. I thought history2 was the best option today.
> > 
> > history2 does not support rich text messages, so no is not a good.
> > 
> >> I just updated my
> >> timetable aiming to convert the XML-based one, and start by preparing the schema
> >> and searching for a good database solution.
> > 
> > Look into mailing list archive. There were already some ideas, so better
> > do not invent wheel from the start, but rather fix design of some
> > existing schemas.
> 
> I had already tried to read previous discussions on the mailing list archive,
> but it is always empty to me ("No messages have been posted to this list yet, so
> the archives are currently empty").

Working link is on GSoC page also with information which you already
should read & process...
https://community.kde.org/GSoC/2017/Ideas#Project:_Chat_history_improvements

Direct link to archive:
https://marc.info/?l=kopete-devel

> >> Do you think converting the SQLite-based plugin into another backend is worth it
> >> (that if SQLite is not the chosen database solution)? I can do it if there is
> >> enough time.
> > 
> > I think SQLite should be fine for metadata. If text messages itself
> > should be in SQLite it would depends on doing some experiments if it is
> > fast enough. Also we need full text search support. SQLite has some
> > plugins for it, but probably different indexes would be better (or
> > not?).
> 
> You are right, better not to reinvent the wheel. But I'm yet to understand where
> we are right now, to be able to move forward. From what I understood, I should
> start from the XML-based plugin, but (probably) use SQLite for storage. That
> sounds like a merge, right?

XML or SQLite is just a storage. For usage you need logic, layer which
can effectively use that storage.

> Maybe I'm missing something obvious to get to the ML archive, maybe someone
> should fix some permission thing. I'd really like to read the previous
> discussions. Anyway, I need help to know which direction to go to.

Before decision for storage it is needed which data to store and what
structure has data...

I dig from archive last idea:

    "id" Integer Primary Key Autoincrement Not Null, --Unique message identifier
    "timestamp" Integer, --When the message was handled
    "message" Text, --HTML containing the message contents
    "protocol" Text Not Null, --Protocol used (Kopete::Protocol::pluginId())
    "account" Text Not Null, --Account used (Kopete::Account::accountId())
    "direction" Integer Not Null, --(Inbound = 0, Outbound=1, Internal=2) (Kopete::Message::MessageDirection)
    "importance" Integer, -- (Low, Normal, Highlight) (Kopete::Message) (Kopete::Message::MessageImportance)
    "contact" Text, -- The local contact used in this message (if applicable). (Kopete::Contact::ContactId()).
    "subject" Text, --If applicable, this will store the subject of the message
    "session" Text, -- Internal session identifier.
    "session_name" Text, -- A human readable name for the session.
    "from" Text, --Internal identifier for the message sender
    "from_name" Text, --Human readable name of the message sender
    "to" Text, --Internal identifier for the message recipient
    "to_name" Text, --Human readable name of the message recipient.
    "state" Integer, --(Unknown = 0, Sending = 1, Sent = 2, Error = 3)
    "type" Integer, --The type of message. (TypeNormal, TypeAction, TypeFileTransferRequest, TypeVoiceClipRequest) (Kopete::Message::MessageType)
    "is_group" Integer Default='0' --If this is set to 1, then we know we are in multi user mode.

Do not know if it match all things, but could be a good starting point.

-- 
Pali Rohár
[email protected]