IMAP4rev2: extended LIST drops \Sent, \Drafts, \Trash and \Junk unless the client sends the RFC 6154 return option that IMAP4rev2 does not define
dennis--- via dovecot <[email protected]>
| Newsgroups | gmane.mail.imap.dovecot |
|---|---|
| Message-ID | <[email protected]> |
SYSTEM INFORMATION
==================
Dovecot version: 2.4.4 (8b687aa65c), Pigeonhole 2.4.4 (3beb331a)
OS: Linux 6.12.76-linuxkit aarch64, Debian 13.5
(official dovecot/dovecot:2.4.4 container image,
digest sha256:723e3392fe16c6fad8ddc605ea767cc01b4bad
9cd9f13eb1dbac15e79c89b2d4)
Host: Docker 29.6.2 (linux/arm64) on macOS 26, Apple
silicon
CPU architecture: aarch64 (arm64)
Filesystem: Docker named volume (overlay2) inside the Docker
Desktop VM. Not NFS.
Feature: experimental IMAP4rev2 (imap4rev2_enable,
mail_utf8_extensions)
Client: our own IMAP client library (Rust), but reproduced
below with openssl s_client so that no client code
is involved
The configuration is the image's own /etc/dovecot/dovecot.conf,
unmodified: nothing of ours is in it. Rather than paste 200 lines of
your own defaults, "doveconf -n" for it is
docker run --rm --entrypoint doveconf dovecot/dovecot:2.4.4 -n
and we will gladly send the output if you would prefer it inline. The
three settings that bear on this report are imap4rev2_enable = yes,
mail_utf8_extensions = yes and mailbox_list_utf8 = yes, all three of
which the image sets itself.
SUMMARY
=======
In an IMAP4rev2 session, an extended LIST (one carrying any RETURN
(...) clause) omits the special-use mailbox attributes unless the
client also sends RETURN (SPECIAL-USE). IMAP4rev2 folded those
attributes into the base protocol and defines no SPECIAL-USE return
option, so a client that implements only RFC 9051 has no defined way
to ask for them, and therefore cannot obtain them from any extended
LIST at all.
The practical effect for such a client is that every folder loses its
role the moment the client asks for anything else -- in our case
RETURN (STATUS (UNSEEN)), to fetch unread counts in the same round
trip. It cannot find the \Sent folder to place a copy of a sent
message in, cannot find \Trash to move a deleted message to, and
renders the mailbox list without role icons or ordering.
We are not asking for a workaround: we already send RETURN
(SPECIAL-USE) on both dialects, since Dovecot advertises the
SPECIAL-USE capability in a rev2 session too. This report is about
what a strictly RFC 9051 client is able to do.
That decision in our client is here, in case it is of interest -- the
comment above it records what we found on your server:
https://github.com/allodia-eu/email-calendar-sync-engine/blob/fa85b881211329480d6738bc7f972d7628cd3b5a/crates/provider-imap/src/capability.rs#L169
REPRODUCTION
============
No configuration is required. dovecot/dovecot:2.4.4 ships
imap4rev2_enable = yes and mail_utf8_extensions = yes in its own
/etc/dovecot/dovecot.conf, and its "@mailbox_defaults = english"
creates Drafts/Junk/Sent/Trash with special-use attributes.
docker run -d --name dovecot-stock -p 127.0.0.1:11999:31993 \
-e USER_PASSWORD=stock-test-pw \
dovecot/dovecot:2.4.4
{ printf 'a1 LOGIN [email protected] stock-test-pw\r\n'; sleep 2;
printf 'a2 ENABLE IMAP4rev2\r\n'; sleep 1;
printf 'a3 LIST "" "*"\r\n'; sleep 1;
printf 'a4 LIST "" "*" RETURN (STATUS (UNSEEN))\r\n'; sleep 1;
printf 'a5 LIST "" "*" RETURN (SPECIAL-USE STATUS (UNSEEN))\r\n';
sleep 1;
printf 'a6 LOGOUT\r\n'; sleep 1; } \
| openssl s_client -connect 127.0.0.1:11999 -quiet
TRANSCRIPT
==========
Verbatim; the greeting and the CAPABILITY lines are elided.
S: * ENABLED IMAP4REV2
S: a2 OK Enabled (0.001 + 0.000 secs).
C: a3 LIST "" "*"
S: * LIST (\HasNoChildren \Drafts) "/" Drafts
S: * LIST (\HasNoChildren \Junk) "/" Junk
S: * LIST (\HasNoChildren \Sent) "/" Sent
S: * LIST (\HasNoChildren \Trash) "/" Trash
S: * LIST (\HasNoChildren) "/" INBOX
S: a3 OK List completed (0.001 + 0.000 secs).
--> correct: the role attributes are present
C: a4 LIST "" "*" RETURN (STATUS (UNSEEN))
S: * LIST () "/" Drafts
S: * STATUS Drafts (UNSEEN 0)
S: * LIST () "/" Junk
S: * STATUS Junk (UNSEEN 0)
S: * LIST () "/" Sent
S: * STATUS Sent (UNSEEN 0)
S: * LIST () "/" Trash
S: * STATUS Trash (UNSEEN 0)
S: * LIST () "/" INBOX
S: * STATUS INBOX (UNSEEN 0)
S: a4 OK List completed (0.034 + 0.000 + 0.033 secs).
--> the problem: every role attribute is gone, and IMAP4rev2
gives the client no option to ask for them back
C: a5 LIST "" "*" RETURN (SPECIAL-USE STATUS (UNSEEN))
S: * LIST (\Trash) "/" Trash
S: * STATUS Trash (UNSEEN 0)
S: * LIST (\Sent) "/" Sent
S: * STATUS Sent (UNSEEN 0)
S: * LIST (\Junk) "/" Junk
S: * STATUS Junk (UNSEEN 0)
S: * LIST (\Drafts) "/" Drafts
S: * STATUS Drafts (UNSEEN 0)
S: * LIST () "/" INBOX
S: * STATUS INBOX (UNSEEN 0)
S: a5 OK List completed (0.002 + 0.000 + 0.001 secs).
--> present again, but only via the RFC 6154 return option,
which RFC 9051 does not define
WHY WE BELIEVE a4 IS NOT CORRECT FOR IMAP4rev2
==============================================
1. RFC 9051 section 7.3.1 makes the special-use attributes ordinary
LIST mailbox attributes:
"In addition to the base mailbox name attributes defined above,
an IMAP server MAY also include any or all of the following
attributes that denote 'role' (or 'special-use') of a mailbox.
These attributes are included along with base attributes
defined above."
Appendix E item 2 lists "the list of new mailbox attributes from
SPECIAL-USE [RFC6154]" among what IMAP4rev2 folded in.
2. RFC 9051 section 6.3.9.2 defines the return options for IMAP4rev2
as exactly SUBSCRIBED, CHILDREN and STATUS. There is no SPECIAL-USE
return option. Nor is SPECIAL-USE among the selection options in
section 6.3.9.1, which are SUBSCRIBED, REMOTE and RECURSIVEMATCH;
it survives in RFC 9051 only as an unexplained
C: A002 LIST (SPECIAL-USE) "" "*"
in the NAMESPACE examples in section 6.3.10, which may itself be
worth an erratum.
3. Section 6.3.9 says "a client MUST NOT send an option for which the
server has not advertised support", so a strictly rev2 client will
not invent one.
Taken together, an RFC 9051-only client can obtain the role attributes
on a basic LIST but never on an extended one, which means it cannot
combine roles with RETURN (STATUS ...) -- the single round trip that
RFC 5819 exists to provide. Since RFC 9051 section 6.3.9 also states
that return options "MUST NOT cause the server to report information
about additional mailbox names", but says nothing about removing base
attributes, we read a4 as a case where an attribute defined as base
data is being withheld.
Note the contrast with \HasNoChildren in the same transcript: it also
disappears in a4, but CHILDREN is a return option in RFC 9051, so a
rev2 client can ask for it back. For special-use there is no
equivalent.
WHAT WE BELIEVE CORRECT BEHAVIOUR WOULD BE
==========================================
In a session where IMAP4rev2 has been ENABLEd, include the special-use
attributes in extended LIST responses without requiring RETURN
(SPECIAL-USE) -- that is, treat them as the base mailbox attributes
that RFC 9051 section 7.3.1 defines them to be. The RFC 6154 return
option would remain accepted for IMAP4rev1 sessions.
NOTE ON IMAP4rev1: WE BELIEVE THAT BEHAVIOUR IS CORRECT
=======================================================
The same server on an IMAP4rev1 session behaves identically, and there
we read it as compliant. RFC 6154 section 2 says "The extended LIST
command MAY return SPECIAL-USE attributes even if the client does not
specify the return option", and MAY is not MUST. This report concerns
only IMAP4rev2 sessions, where the way to ask was removed along with
the need to.
CONTACT
=======
Happy to test a patch. The reproduction is a single "docker run", and
we keep both an IMAP4rev1 and an IMAP4rev2 Dovecot fixture in our CI,
so a fix can be regression-tested on both dialects.
_______________________________________________
dovecot mailing list -- [email protected]
To unsubscribe send an email to [email protected]