Re: SmtpAppender in log4j2, plus how to reconfigure everything

Piotr,

On 6/1/25 4:47 AM, Piotr P. Karwasz wrote:
> Hi Chris,
> 
> On 30.05.2025 20:23, Christopher Schultz wrote:
>> For my purposes, I need to be able to:
>>
>> 1. Attach custom headers to SMTP messages.
>>
>> I did this by creating a custom subclass[1] of SMTPAppender that
>> overrides activateOptions and sets whatever headers I need on the SMTP
>> message.
>>
>> In log4j2, SmtpAppender is final, so this is no longer possible.
>>
>> Are there any suggestions for how to add custom headers to messages sent
>> via SmtpAppender?
> 
> 
> Since the HTTP appender[1] already supports the configuration of custom
> headers using nested <Property> elements, it would make sense to add
> similar support to the SMTP appender as well. I've gone ahead and
> created Issue #3704[2] to track this enhancement. Feel free to submit a
> PR if you're interested in implementing it.

Thanks for doing that. I'll definitely track that, and also see if I'm 
able to create a useful PR.

> As for extending the `SmtpAppender`, most Log4j Core 2 appenders follow
> a similar architectural pattern:
> 
> - The `Appender` itself is fairly minimal and mainly responsible for
> exposing a builder that handles configuration options. This part
> typically doesn't require extensibility.
> 
> - The actual work is done by a `Manager`, which is reference-counted to
> allow reuse across multiple `Configuration`s and even `LoggerContext`s.
> This ensures that shared resources like a Java Mail session aren’t
> closed during reconfiguration.
> 
> For SMTP, Log4j uses an abstract `MailManager` to support both Java EE
> and Jakarta EE environments. Each implementation handles the protocol
> specifics while sharing common logic through the abstract base.
> 
> [1]
> https://logging.apache.org/log4j/2.x/manual/appenders/network.html#HttpAppender
> [2] https://github.com/apache/logging-log4j2/issues/3704
> 
>> 2. Don't send log messages as individual SMTP messages, send all at once.
>>
>> I did this by creating a custom[2] TriggeringEventEvaluator that simply
>> delays triggering until it gets to some high threshold.
>>
>> There doesn't seem to be anything at all like that in log4j2.
>>
>> Are there any suggestions for how to get multiple log events per SMTP
>> message?
> 
> This functionality is actually already supported in Log4j Core, although
> the behavior may not be immediately intuitive.
> 
> Like most Log4j Core appenders, the `SmtpAppender` accepts a nested
> filter element[2]. However, unlike other appenders, the SmtpAppender
> does not use the filter to drop log events. Instead, it uses the filter
> to determine how events are handled:
> 
> - Events that do not pass the filter (i.e., the filter returns DENY) are
> still captured and stored in a cyclic buffer. These events provide
> context and will be included in the body of the email if an email is
> triggered later.
> 
> - Events that do pass the filter (i.e., the filter returns ACCEPT or
> NEUTRAL) trigger the sending of an email. When this happens, the content
> of the email includes not just the triggering event but also the
> buffered context events that were previously collected.
> 
> For more details, see the documentation on the SMTP appender here[4].
> 
> [3] https://logging.apache.org/log4j/2.x/manual/filters.html
> [4]
> https://logging.apache.org/log4j/2.x/manual/appenders/network.html#SmtpAppender

This is also very helpful. It sounds like I need a filter that simply 
does not pass anything (i.e. filters everything). That would cause the 
SmtpAppender to hold everything together. At the end (this is a 
scheduled process which starts and ends, and I want to deliver at the 
end of the process), how does one indicate to the appender that it's 
time to "let it all go"? I don't necessarily want to log a message like 
"done!" because I don't want that actually in the output. Will the 
SmtpAppender automatically send on shutdown? I am currently calling 
LogManager.shutdown() at the end of the process (log4j 1.x).

>> 3. Re-configure the logging system to enable TRACE logging (or really
>> anything kinda like this).
>>
>> The idea is that there is an on-disk file with the typical configuration
>> (e.g. production-style low-message-output config) but then I'd like to
>> pick out a specific logger or appender, change its threshold, and have
>> that configuration become effective across the whole logging system, at
>> runtime, without re-starting the JVM or anything like that.
> 
> This is actually one of the core strengths of Log4j Core 2. Unlike Log4j
> 1.x or Logback, Log4j Core 2 supports seamless, lossless
> reconfiguration—even during high-throughput logging. No log events are
> lost in the process.
> 
> To enable this feature, simply set the `monitorInterval`[5] attribute on
> the `<Configuration>` element to a positive integer. This tells Log4j to
> poll the configuration source (such as a `file:` or `https:` URI) every
> n seconds, automatically reloading the configuration if it detects any
> changes.
 >
> Behind the scenes, Log4j's Manager mechanism (as mentioned earlier)
> ensures that shared resources, such as file handles or SMTP sessions,
> remain open and consistent across reconfigurations. For the SMTP
> appender it means that the internal cyclic buffer holding context log
> events is preserved across configuration reloads—meaning you won’t lose
> the context of an `ERROR` event just because the config was updated.
> 
> Looking ahead, starting with version 2.25.0, we’re introducing a new
> `MonitorResources` mechanism inspired by the `WatchedResources` you have
> in Tomcat. This will allow you to monitor additional resources (e.g.,
> keystores, certificate files) for changes and trigger a reconfiguration
> if they are modified—even if the main config file itself remains untouched.

I don't want to modify any files, here, if possible. The idea is to load 
the "proper" configuration from the file as usual, but then be able to 
enable or disable e.g. TRACE logging for specific loggers and/or 
appenders as necessary.

Is there any way to do this kind of thing programmatically? I can see in 
the Solr Dashboard that (a) log4j2 is being used and (b) appears to 
allow the user to adjust the log level for all loggers through that 
interface. I haven't gone into the code to figure out how it's done, but 
it leads me to believe that everything can be done ephemerally in a 
running JVM without writing any files or anything like that.

This is the kind of thing that I'm looking for.

Thanks!
-chris