Re: Dovecot 2.4 doveadm_worker_count corrupts tab-formatted output

Hi Aki, Dovecot developers

so this is a bug in dovecot 2.4 which prevents iteration over 1000s of 
users with doveadm and -A flag. The workaround as identified by Philip 
is to do iteration outside dovecot and then feed that into doveadm with 
-u flag.

Maybe it's not such a big issue for Dovecot CE where user bases may be 
low enough not to run into it. Could be of more of interest for large 
users bases and Dovecot Pro. The effect is that doveadm quota recalc and 
index does not terminate correctly.

It was working on dovecot 2.3 and the code hasn't really undergone that 
much change.

My assumption is that in auth/auth-master-connection.c, in the 
master_input_list_callback() function where there is a call 
o_stream_flush(ctx->conn->output), that this call no longer appears to 
reduce the o_stream_get_buffer_used_size under the MAX_OUTBUF_SIZE limit 
and so get into a case with the code did not handle correctly.

This patch seems to fix it in Dovecot 2.4 by always 
calling userdb_blocking_iter_next(ctx->iter) regardless of used buffer 
size. I admit though that this may not be the right fix, if the 
difference in the way  o_stream_flush works is the root cause.

--- auth/auth-master-connection.c.orig    2026-03-08 18:06:28.025317016 
+0000
+++ auth/auth-master-connection.c    2026-03-08 18:07:08.590631101 +0000
@@ -570,10 +570,9 @@
          master_input_list_finish(ctx);
          return;
      }
-    if (o_stream_get_buffer_used_size(ctx->conn->conn.output) < 
MAX_OUTBUF_SIZE)
-        userdb_blocking_iter_next(ctx->iter);
-    else
+    if (o_stream_get_buffer_used_size(ctx->conn->conn.output) >= 
MAX_OUTBUF_SIZE)
          o_stream_uncork(ctx->conn->conn.output);
+    userdb_blocking_iter_next(ctx->iter);
  }

  static int master_input_list(struct auth_master_connection *conn,

John

On 01/03/2026 16:36, John Fawcett via dovecot wrote:
>     Hi
>
>     I can confirm the issue of these iteration operation not completing for
>     1000's of users is linked to:
>
>     MAX_OUTBUF_SIZE defined in auth/auth-master-connection.c
>
>     I made it 1024000 and managed to complete quota recalc for 10000 users.
>     But that's not going to be a solution, since whatever it is set to
>     provides a hard coded limit to how many users can be iterated for quota
>     get, quota recalc and index operations. Surprisingly it is not a
>     completely deterministic limit, since sometimes more users get processed
>     than other times. I was looking for a time based limit before I thought of
>     looking at the buffer size.
>
>     The strange thing is that the current value hasn't changed in more than 20
>     years if ever, so that does not explain of itself why this issue arose now
>     in 2.4.
>
>     #define  MAX_OUTBUF_SIZE  (1024*50)
>
>     The likely explanation is that exceeding the buffer size is now being
>     handled differently to before.
>
>     John
>
_______________________________________________
dovecot mailing list -- [email protected]
To unsubscribe send an email to [email protected]