Re: bincdrop status / Binc 1.5.x

Andreas Aardal Hanssen wrote:
> On Mon, 8 Aug 2005, Jerry Lundström wrote:
> 
>>I agree that one should make as little as possible for the most value... but
>>today binc isnt really there. Thats why I want to use C++/STL alot more.
> 
> The STL is not a solution to any design problems; it's simply a set of 
> template classes. It's very important not to over engineer here. I don't 
> think the std streams (in particular) are any good at all. Especially 
> considering how poor support there is for modern STL/std classes in older 
> compilers.

I don't see the need to design io objects when there is already a 
standard and if its old compilers your worried about why not use C 
insteed? (later edit) No don't answer that, hehe, too early in the 
moring and not enought coffie.
What systems/compiles today do we know of that are using C++ and doesn't 
have STL?

>>The problem with today API is that its very statically orianted around 
>>Maildir format and filesystem.
>>The Depot class should be the class handling a depot (doh!) and 
> 
> What do you mean with the "doh!"? Please explain.

This is what I think about the classes that exist today:

Depot : Super class for depositories, diffrent type of mailbox formats.

Mailbox : Super class for a mailbox contained inside a Depot.

Message : Super class for a message contained inside a Mailbox.

The 'doh!' was just an expression of the obvious explaination of the 
depot class.

>>delivering a Mailbox class. In Maildir++ those Mailboxes would be named 
>>INBOX INBOX.subfolder etc and in IMAPdir INBOX/ INBOX/subfolder. There 
>>should never be functions like filenameToMailbox etc because that only 
>>consentrats on a filesystem.
> 
> The Depot is an abstraction for storing an IMAP mailstructure on the file 
> system. It works splendidly in that sense, and there's no reason to design 
> it to do anything else. It already acts as the authority for creating 
> Mailbox objects and everything that has to deal with mailboxes. Of course 
> the filenameToMailbox functions could be designed differently, but the 
> only reason for totally changing them would be if Depot should start 
> supporting depot types that are not file based, and there are no such 
> depots today. I do not design software to support what <10% of our users 
> will use actively.

I understand that you dont want to spend time making super freaky 
storing layers that noone would use. It's just bad, you infect the 
codebase with possible expliots that leaves your with more to maintain.

But!, the abstractive design today is very good, and if it can be change 
to just that... an abstaction... it could be used for so much more.

Getting bincdrop to work with the current code will have to change alot 
of the current code since its not really abstact. It's very tied to the 
fact that it uses an IMAP protocoll and accesses an Maildir depot.

>>A Mailbox should later deliver Message objects that are based on an ID, 
>>beging the filename or the IMAP UID isnt what matters, it should only be 
>>an unique identifier.
> 
> I seriously don't understand what the difference is between a UID and a 
> unique identifier. Please elaborate how this is different from today's 
> approach.

If you want an object orianted abstract Mailbox/Message, the mailbox and 
the message itself should not know about IMAP UID's. That should be 
handled inside the IMAP server code when it connects the Message Name/ID 
to the cached IMAP UID.

>>The point of sliming down the API to this is to make it workable in 
>>situations as bincdrop where printing the header isnt importent or 
>>having a selected mailbox.
> 
> I'm sorry, but I don't quite understand how today's API, which allows you 
> to search and print any part of all headers in any message, prevents 
> bincdrop from printing the header... Store the message on disk, use 
> today's API, and voila.

Yes and no, today's API doesnt let you mess with the MimeDocument from a 
Message. The MimeDocument inside a MaildirMessage is private and all the 
functions provided by the MaildirMessage is really for printing the 
MimeDocument in a IMAP protocoll specific way.

I also had to rewrite almost all functions consering creating and 
storing new messages since that also was just made to work within the 
limits of an IMAP operation.

>>The open() close() flush() is just a common name of opening closeing and 
>>flushing changes to all the objects that I saw fitting to use.
> 
> Well I just found them confusing ;-). Mailboxes are selected and closed, 
> and depots are iterated over.

Well, I dont see why an Mailbox object should know if its selected or 
not. In an objective orianted approach that would more likly reside 
inside the IMAP server object.

>>The Maildir::Mailbox is just wrong =) I dont have a good way of explaing 
>>that in C++ but I would like to se each type of Depot (being Maildir++ 
>>or IMAPdir or mbox whatever) in its own namespace.
> 
> There's no need to use namespaces like that in Binc; we already have 
> classes that group the concepts together. Nesting classes is in general a 
> very confusing design pattern that applies only to a very limited set of 
> programming languages. So I think we should avoid that.

I was just thinking it would be nice to reside the Maildir++ related 
objects inside its own namespace to not dilute the general namespace.

>>>This is what Binc does today, and it does it quite fast and I/O
>>>efficiently. But the API could be better ;-) hehe.
>>
>>I am not saying we should change the mime parsing code, I just want to 
>>use C++/STL streams to transport the data and not be filebased.
>>If the API was using streams it would be possible to move mail from one 
>>type of depot to another without much code because the sending depot 
>>delivers a stream that you can just put into the receveing depot.
>>I really don't think there is alot of work to be done to make the API 
>>very object orianted and usable in alot other applications and im 
>>willing to do most part of the work but there must be a new API that 
>>everyone can agree on.
> 
> OK, but please don't do anything yet. Firstly, Binc IMAP is designed to 
> use file system based Depots, but mailboxes can be of any type. That 
> simplifies the design radically, and is one of the main reasons for why 
> Binc is fast. Don't pass all data through streams before doing performance 
> measures that show that we're as fast as today. I speeded up the MIME 
> parser in 1.3 to make us even faster than 1.2; not to compensate for 
> degraded performance due to the introduction of std streams.

Ofcouse I'm not gonna start running the code in my dryer to mess it all 
up, there is so much to talk about before even starting to design.

I don't think passing data with streams as buffer is the way, I was more 
looking for a way to pass a stream that can be used to get the data 
inside the MIME object without the MIME object knowing that its a file 
or something else.

> For Binc 1.3, we have two main goals and several smaller ones. The main 
> goals are i18n and SORT/THREAD, and one of the big smaller goals is 
> indexing. I understand that bincdrop could make use of a different API, 
> but bincdrop is a separate tool, and it doesn't need Binc IMAP to operate. 
> And what I'm trying to say is that Binc IMAP should not _necessarily_ 
> change because of bincdrop. If you need to change what works fine today to 
> make bincdrop work, then maybe bincdrop should change instead.

That makes it perfectly clear that bincdrop needs its own package.

> All our major feature requests for 1.3 can be implemented without changing 
> the design of the server radically. When I'm talking moderate API 
> redesign, I'm all for refactoring the API to make it easier to work with 
> and easier to maintain and extend. But I really don't see the value in 
> doing it all differently at this point. And in particular, I don't see the
> value of introducing streams.

To me, I don't see the value in making your own IO streams therefor im 
all for using what exists.
And I'm not talking about redesign everything from scratch, the 
Depot/Mailbox/Message is a very nice approach, but I'm questioning if 
functions like printBody should really exist inside the Message object.

Regarding streams or not, I'll try to make a little mime handling object 
  from a object orianted approach so we can have something to look at, 
discuss and preformance test.

As a side note, we are now running binc on our imap servers and at this 
very moment there are 1064 clients connected and the load isnt noticable.

I'm very happy about the development of binc and excited to be a part of it!

Keep up the good work Andy!

-- 
Jerry Lundström, System Developer
Section for IT and Media, Stockholms University, Sweden
+46 (0)8 16 19 99 / http://www.it.su.se