Dovecot 2.4 doveadm_worker_count corrupts tab-formatted output

Hi there

On a Dovecot 2.4.2 (Debian Bookworm) server with over 1300 users, I am using the following config to retrieve quota usage information with doveadm:

---------------------------------------

quota "User quota" {
}

mail_plugins = quota acl mail_log notify

userdb sql {
  iterate_query = SELECT username AS user, CONCAT(quota, 'M') AS quota_storage_size FROM mailaccounts WHERE active = 1

  query = \
    SELECT '/var/vmail/%{user | domain}/%{user | username}' AS home, 'vmail' AS uid, 'vmail' AS gid, CONCAT(quota, 'M') AS quota_storage_size \
    FROM mailaccounts \
    WHERE username = '%{user}' AND (active = 1 OR transport_suspended = 0)
}

---------------------------------------

Initially, I had to run `doveadm quota recalc -A` to recalc quotas and populate the `count` quota driver which is enabled by default for "User quota".
But still, it took roughly 20ms per single user or 15-20s for all users:

$ time doveadm -f tab quota get -u [email protected]
real	0m0.020s
$ time doveadm -f tab quota get -A
real	0m16.800s

Now, to greatly speed this up, I have introduced parallel processing:

doveadm_worker_count = 4

Like this, `doveadm -f tab quota get -A` runs approx 10x faster, done in 1.5 - 2.5s
But there is one major drawback which I assume to be a bug:

# sane output without `doveadm_worker_count = 4` => always 6 fields
[email protected]	User quota	STORAGE	29	204800	0
[email protected]	User quota	MESSAGE	1	-	0

# broken output with `doveadm_worker_count = 4` => sometimes 7 fields
[email protected]	User quota	STORAGE	29	2048	00		0
[email protected]	User quota	MESSAGE	1	-	0

Somehow, the tab formatter does not play nicely together with parallel processing. On some output, the value gets split apart by an extra tab, e.g. '2024<TAB>00' in above example. Can you explain this corrupted data? Some kind of "output interleaving" seems to happen when running parallel workers.
If I switch to json output (`doveadm -f json quota get -A`), this does not seem to happen and I always get valid JSON output, even with `doveadm_worker_count = 4`.

Cheers,
Philip

_______________________________________________
dovecot mailing list -- [email protected]
To unsubscribe send an email to [email protected]