Re: [patch] proposed patch for managing timeouts on doveadm iteration with -A

Thanks for this Timo, I'll take a look and post my results asap.

On 05/08/2026 23:35, Timo Sirainen via dovecot wrote:
> On 26. Jul 2026, at 0.17, John Fawcett via dovecot <[email protected]> wrote:
>> Hi
>>
>> having further investigated the timeouts mentioned in the previous thread, I am submitting this patch as a potential solution.
>>
>> Basically with -A doveadm parameter, the 155 second timeout is applied to the whole end to end transaction. With -F parameter the 155 second timeout is applied to each single user operation. Changing the logic of -A to follow -F looks a bit invasive. So this patch does more or less the same thing, i.e. giving each user operation 155 seconds timeout, done in a different way. Whenever another username is returned, the timeout is reset.
>>
>> PS I got help from Claude on this.
> Some further improvements via Fable - does this still fix your problems?
>
> commit 9deefd88b37d4e9bfecdf6bd1c9fa8a3f9d6b100
> Author: Timo Sirainen <[email protected]>
> Date:   Wed Aug 5 21:33:42 2026 +0000
>
>      lib-auth-client: Extend LIST request timeout whenever a username is returned
>      
>      The auth lookup timeout (155 seconds by default) was applied to the whole
>      LIST request, i.e. the entire user enumeration. With large installations
>      listing all users can easily take longer than that, causing e.g.
>      doveadm -A to fail with "Auth server request timed out".
>      
>      Reset the request's timeout whenever another username is returned. This
>      turns the timeout into a "no progress for N seconds" stall detector,
>      matching how independent short-lived per-user requests (e.g. as used when
>      iterating -F user files) never accumulate a single deadline across the
>      whole run.
>      
>      The timeout handling assumes that requests expire in creation order, so
>      extending is only allowed while the request is alone on its connection.
>      This is always the case for LIST, which is asserted at init already.
>      
>      Based on code by John Fawcett
>
> diff --git a/src/lib-auth-client/auth-master-private.h b/src/lib-auth-client/auth-master-private.h
> index 770195c2dc..10986ee155 100644
> --- a/src/lib-auth-client/auth-master-private.h
> +++ b/src/lib-auth-client/auth-master-private.h
> @@ -89,6 +89,7 @@ struct auth_master_connection {
>   
>   unsigned int
>   auth_master_request_get_timeout_msecs(struct auth_master_request *req);
> +void auth_master_request_extend_timeout(struct auth_master_request *req);
>   
>   void auth_master_request_send(struct auth_master_request *req);
>   int auth_master_request_got_reply(struct auth_master_request **_req,
> diff --git a/src/lib-auth-client/auth-master-request.c b/src/lib-auth-client/auth-master-request.c
> index cc6feba483..ef153ac18a 100644
> --- a/src/lib-auth-client/auth-master-request.c
> +++ b/src/lib-auth-client/auth-master-request.c
> @@ -31,6 +31,19 @@ auth_master_request_get_timeout_msecs(struct auth_master_request *req)
>   	return (unsigned int)(msecs < 0 ? 0 : msecs);
>   }
>   
> +void auth_master_request_extend_timeout(struct auth_master_request *req)
> +{
> +	struct auth_master_connection *conn = req->conn;
> +
> +	/* The timeout handling assumes that requests expire in the order
> +	   they were created, so the deadline of the oldest request must not
> +	   be pushed past the deadlines of newer requests. */
> +	i_assert(conn->requests_count == 1);
> +
> +	req->create_stamp = ioloop_timeval;
> +	auth_master_connection_update_timeout(conn);
> +}
> +
>   static void auth_master_request_remove(struct auth_master_request *req)
>   {
>   	struct auth_master_connection *conn = req->conn;
> diff --git a/src/lib-auth-client/auth-master.c b/src/lib-auth-client/auth-master.c
> index 4c19cfbcc3..5414f723c5 100644
> --- a/src/lib-auth-client/auth-master.c
> +++ b/src/lib-auth-client/auth-master.c
> @@ -1381,6 +1381,17 @@ const char *auth_master_user_list_next(struct auth_master_user_list_ctx *ctx)
>   	if (username == NULL)
>   		return NULL;
>   
> +	/* A LIST request is a single long-lived, multi-reply request that
> +	   stays alive for the entire user enumeration, which for large
> +	   installations can take far longer than the normal auth lookup
> +	   timeout allows for a single request. Since we just made progress
> +	   (another username was returned), push the request's deadline out
> +	   from now instead of leaving it anchored to when the LIST request
> +	   was first created. This turns the timeout into a "no progress for
> +	   N seconds" stall detector for listing. */
> +	i_assert(ctx->req != NULL);
> +	auth_master_request_extend_timeout(ctx->req);
> +
>   	e_debug(ctx->event, "Returned username: %s", username);
>   	return username;
>   }
>
> _______________________________________________
> dovecot mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
_______________________________________________
dovecot mailing list -- [email protected]
To unsubscribe send an email to [email protected]