Re: Input stream limit exceeded on elasticsearch

Hello Mahia,

You are right we set up a 100MB limit on the mails indexed in 
ElasticSearch, which is already very generous: such mails should rather 
not exist in the first place.

Under the hood we uses MimeConfig.PERMISSIVE for the parsing in 
MimePartParser.

While I am not sure this is a priority we could welcome a code patch to 
make it configurable.

Best regards,

Benoit

On 01/05/2023 03:34, Mihai Zamfira wrote:
> Hi,
> I'm trying to configure an Apache James 3.7.4 that uses Cassandra, ElasticSearch and OpenLDAP. When synchronizing emails, I get the following error
>
> 23:26:23.748 [WARN ] o.a.j.m.e.v.e.ElasticSearchListeningMessageSearchIndex - Indexing mailbox Sent-b947fde0-e3a7-11ed-ade3-e76310c4a198 of user [email protected] on message MessageUid{uid=2710} without attachments
> java.io.IOException: Input stream limit exceeded. Limit was 104857600 and position 104857600
> 	at org.apache.james.mime4j.io.LimitedInputStream.enforceLimit(LimitedInputStream.java:39)
> 	at org.apache.james.mime4j.io.LimitedInputStream.read(LimitedInputStream.java:51)
> 	at java.base/java.io.FilterInputStream.read(FilterInputStream.java:107)
> 	at org.apache.james.mime4j.stream.MimeEntity.advanceToBoundary(MimeEntity.java:393)
> 	at org.apache.james.mime4j.stream.MimeEntity.advance(MimeEntity.java:331)
> 	at org.apache.james.mime4j.stream.MimeTokenStream.next(MimeTokenStream.java:374)
> 	at org.apache.james.mailbox.elasticsearch.v7.json.MimePartParser.parse(MimePartParser.java:68)
> 	at org.apache.james.mailbox.elasticsearch.v7.json.IndexableMessage$Builder.instantiateIndexedMessage(IndexableMessage.java:101)
> 	at org.apache.james.mailbox.elasticsearch.v7.json.IndexableMessage$Builder.build(IndexableMessage.java:72)
> 	... 64 common frames omitted
> Wrapped by: java.lang.RuntimeException: java.io.IOException: Input stream limit exceeded. Limit was 104857600 and position 104857600
> 	at org.apache.james.mailbox.elasticsearch.v7.json.IndexableMessage$Builder.build(IndexableMessage.java:74)
> 	at org.apache.james.mailbox.elasticsearch.v7.json.MessageToElasticSearchJson.convertToJson(MessageToElasticSearchJson.java:67)
> 	at org.apache.james.mailbox.elasticsearch.v7.events.ElasticSearchListeningMessageSearchIndex.generateIndexedJson(ElasticSearchListeningMessageSearchIndex.java:165)
> 	at org.apache.james.mailbox.elasticsearch.v7.events.ElasticSearchListeningMessageSearchIndex.lambda$add$0(ElasticSearchListeningMessageSearchIndex.java:158)
> 	at reactor.core.publisher.MonoCallable.call(MonoCallable.java:92)
>
>
> It is clearly that is a limit of 100Mb in the integration stream with ElasticSearch, but I can't find how can increase it.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
>
>