[jira] [Commented] (JAMES-4159) EventBus: improve slow listener handling

    [ https://issues.apache.org/jira/browse/JAMES-4159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18050304#comment-18050304 ] 

Matthieu Baechler commented on JAMES-4159:
------------------------------------------

If we know some listeners have different behaviors, why not making delivery system knows?

For example, we could have a way to group listeners explicitly. We could group fast listeners but not slow ones.

What do you think?

> EventBus: improve slow listener handling
> ----------------------------------------
>
>                 Key: JAMES-4159
>                 URL: https://issues.apache.org/jira/browse/JAMES-4159
>             Project: James Server
>          Issue Type: Improvement
>          Components: eventbus
>            Reporter: Benoit Tellier
>            Priority: Major
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> h3. Context
> At LINAGORA we developped some custom listeners responsible of doing AI related tasks, either querying or feeding AI models. Mailbox listeners are a good fit because they are not blocking / delaying mail reception.
> The EventBus works the following way: it publishes a message onto RabbitMQ, and initially is consummed once by a James node that would execute all listeners. Using QOS we are consuming up to 10 messages simultaneously per node. Failed messages would then be re-published to individual per-listener retry queue.
> This "bundling" is done so that we limit the chatter of deserialization + wiring of the event bus, where each listener would have needed those operations, with separated execution. The amount of CPU time dedicated to those operations was significant ( > 40%).
> However this bundling means we are not efficient handling "slow" listeners which at scale can sinificantly distrurb and delay event treatment.
> h3. Proposal
> Be able to configure a timeout on the initial eventbus run, for each listener.
> That way, slow listeners would be individually retried.
> In `listeners.xml`:
> {code:xml}
> <listeners>
>   <executeGroupListeners>true</executeGroupListeners>
>   <!-- New block -->
>   <qos>20</qos>
>   <initialExecutionTimeout>30s</initialExecutionTimeout>
>   <executionTimeout>120s</executionTimeout>
>   <!-- -->
>   <listener>
>     <class>org.apache.james.mailbox.cassandra.MailboxOperationLoggingListener</class>
>   </listener>
> </listeners>
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)