Re: Re: A Question about KafkaAppender

--0000000000004e3122062cb0794e
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

What you describe is exactly what the burst filter is designed for. Can you
explain why it doesn=E2=80=99t work for you?

I will soon write a detailed response on resurrecting Kafka appender.

Op ma 27 jan 2025 om 14:41 schreef Yuepeng Pan <[email protected]>

> Thanks Volkan for the comments and help.
>
>
>
>
> It sounds like neither of the two methods mentioned above can meet
>
> the business scenario requirements:
>
> We just want the Kafka appender to discard data only when its output
>
> rate is lower than the log production rate.
>
>
>
>
> > If you are actively using it, either consider migrating to an
> alternative,
>
> > or step up as a maintainer, please.
>
>
>
>
> I'm willing to make some contributions to the Kafka appender to the best
> of my ability.
>
> In addition, I am curious:
> - if I want to support the feature where the Kafka
>
> appender can discard logs when the output rate is lower than the log
> generation rate,
>
> what specifications or rules should I follow to advance this feature?
> - Is this feature reasonable in the eyes of the community's users and
> developers?
>
>
>
>
> Thank you very much.
>
>
>
>
> Best,
>
> Yuepeng
>
>
>
>
>
>
>
>
>
>
>
> At 2025-01-27 17:27:03, "Volkan Yaz=C4=B1c=C4=B1" <[email protected]> wrote:
> >Hello Yuepeng,
> >
> >If it is okay to drop log events when the appender isn't keeping up, you
> >can use a burst filter
> ><https://logging.apache.org/log4j/2.x/manual/filters.html#BurstFilter>.
> If
> >your burst/congestion periods are temporary and you don't want to lose
> >events, you can consider employing an async. appender
> ><
> https://logging.apache.org/log4j/2.x/manual/appenders/delegating.html#Asy=
ncAppender
> >
> >as a buffer.
> >
> >Note that the Kafka appender
> ><
> https://logging.apache.org/log4j/2.x/manual/appenders/message-queue.html#=
KafkaAppender
> >
> >sadly
> >needs some love. Due to lack of community interest and maintainer time, =
it
> >is planned to be dropped in the next major release, i.e., Log4j 3. If yo=
u
> >are actively using it, either consider migrating to an alternative, or
> step
> >up as a maintainer, please.
> >
> >Kind regards.
> >
> >On Sun, Jan 26, 2025 at 12:09=E2=80=AFPM Yuepeng Pan <panyuepeng@apache.=
org>
> wrote:
> >
> >> Hi, masters..
> >>
> >>
> >> Recently, I have enabled the Kafka appender in certain scenarios to
> >> collect logs, but we encountered an issue:
> >> When the log generation speed exceeds the write speed of Kafka,
> >>  it negatively impacts the processing speed of core business logic
> because
> >> the high-frequency log output is embedded within the core business
> logic.
> >>
> >>
> >> May I know is there any available parameter for optimizing this issue?
> >>
> >>
> >> Thank you~
> >>
> >>
> >> Best,
> >> Yuepeng Pan
>

--0000000000004e3122062cb0794e--