Re: Dovecot 2.4 doveadm_worker_count corrupts tab-formatted output
Philip Iezzi via dovecot <[email protected]>
| Newsgroups | gmane.mail.imap.dovecot |
|---|---|
| Message-ID | <[email protected]> |
> On 27 Feb 2026, at 01:24, John Fawcett via dovecot <[email protected]> wrote: > > > On 26/02/2026 21:20, Philip Iezzi via dovecot wrote: >> ... >> Hi John >> >> Thanks for looking into this. Yes, those were the last lines of `doveadm -Dv quota get -A` debug output, all related to the same user - I have simply redacted the user with [email protected], but did not change anything else / did not remove any lines. And yes, the problem always occurs on another user and is definitely not related to a specific user. >> I don't really understand the logic here - so maybe it's just some kind of chained request that triggers the next one (2802712577) via `userdb list`. >> >> I also don't quite remember why I added `quota_storage_size` to the iterate_query. Somehow I struggled to get quotas working during Dovecot 2.3 => 2.4 config migration. I have removed it now and all still seems to be working with: >> >> iterate_query = SELECT username AS user FROM mailaccounts WHERE active = 1 >> >> Thanks for this input. But this doesn't make a difference. I always get the same Error: >> >> $ doveadm -f json quota get -A >/dev/null; echo $? >> doveadm([email protected]): Error: auth-master: userdb list: User listing returned failure >> doveadm: Error: cmd quota get: Failed to iterate through some users >> 75 >> >> correlating with exactly 1 such line in mail.log: >> >> dovecot: auth: Error: auth-worker: Aborted LIST request for *: Shutting down >> >> Here's the thing: This only happens on my mail server with 2200 users, not on the other with "only" 1200 users, both running the exact same config/versions. >> I have now invented this nice funky workaround to completely do without depending on the iterate_query. I don't even need to grab a subset of the users / do any sharding, just pipe all users into `-F`: >> >> $ mysql --quick -srN maildb -e 'SELECT username FROM mailaccounts WHERE active=1' | doveadm -f json quota get -F - >> >> It always works, always gives me the full data (checked with `wc -c`, while `-A` always broke in the middle and never returned the quotas for all users), and it never provokes a crashed auth-worker, above Error no longer showing up. >> >> But I'd like to emphasize, that on Dovecot 2.3, I was running `doveadm quota get -A` for years, without ever running into this issue. It ran as a cache warmup job from my application and pulled all quota usages from Dovecot server, every 4 minutes! I know this is probably not recommended and I could easily reduce it to run only every 15mins or even less frequently. But no matter how often I run it on that server with 2200 users, it always runs fast enough, but always breaks since upgrading to Dovecot 2.4 >> >> Cheers, >> Philip > > Hi Philip, it really does look like you're hitting some kind of limit. Potentially it could be a mysql timeout but seems unlikely you would have changed mysql_read_timeout from its default 30s. > I wonder if it's an issue only for quota or whether it happens on other iteration queries, for example doveadm user '*'. > John Yes, I am definitely running into some kind of limit. It's not on MySQL level - tuned it a lot for high performance and connection hammering, and never found any error in MySQL log. Above workaround is now running for a full day, pulling all user quotas every 4mins, without running into this issue again. As `doveadm quota get -A` was always running fine on Dovecot 2.3 and I can now (under Dovecot 2.4.2) retrieve all quotas by piping all users into `-F`, it should definitely be possible to fix/optimize the `-A` iterate_query for mailservers with larger user base. @Aki did you guys experience this issue and can I provide you any more information to help you get this fixed? I can definitely live with my workaround, just hope others are not struggling with it and wasting so much time as I did. Best regards, Philip _______________________________________________ dovecot mailing list -- [email protected] To unsubscribe send an email to [email protected]