Re: certificate errors result in excessive logging.
Balazs Scheidler <[email protected]> Mon, 21 Oct 2024 22:32:54 +0200
| Newsgroups | gmane.comp.syslog-ng |
|---|---|
| Message-ID | <CAKcfE+ZtOZysV7Nd=NOJgxx8X0wxOVv9CZeYVvru+9cW6VKJJQ@mail.gmail.com> |
--===============8937749272506296314== Content-Type: multipart/alternative; boundary="00000000000055f18c0625028fb0" --00000000000055f18c0625028fb0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable you can always use rate-limit: https://axoflow.com/docs/axosyslog-core/chapter-routing-filters/filters/ref= erence-filters/filter-rate-limit/ Balazs On Mon, Oct 21, 2024 at 9:58=E2=80=AFPM Evan Rempel <[email protected]> wrote= : > I don't want to filter them out completely. This still represents an issu= e > and the client is unable to log the issue to the central > log/SIEM/monitoring server. I just don't want to fill up my disks with > message repeatedly. I only need to "hear about it" once every hour (or so= me > other throttled period). > > Technically there are lots of ways to leverage syslog-ng configurations t= o > address this but I thought that since this would be an issue for anyone > that has a lot of syslog clients and then has an issue with their > certificate, I thought there might be a cleaner and more simple way to > address than the correlation/throttling/filtering tools of syslog-ng. > > For me, the log rate is so large that if I have only a small % of my > clients that have this issue, they will take the service off-line for all > of the other clients. > > -- > Evan > ------------------------------ > *From:* syslog-ng <[email protected]> on behalf of > Balazs Scheidler <[email protected]> > *Sent:* October 21, 2024 12:47 PM > *To:* Syslog-ng users' and developers' mailing list < > [email protected]> > *Subject:* Re: [syslog-ng] certificate errors result in excessive logging= . > > You don't often get email from [email protected]. Learn why this is > important <https://aka.ms/LearnAboutSenderIdentification> > hi, > > I literally submitted a patch this morning to mute #3 from the list above= . > https://github.com/axoflow/axosyslog/pull/352 > > As for the other two, you can filter these out from your internal() sourc= e > using regexps. > Bazsi > > On Mon, Oct 21, 2024 at 9:30=E2=80=AFPM Evan Rempel <[email protected]> wro= te: > > I am using tls configuration with a locally signed certificate. This mans > that I have to configure a custom root CA on to all client systems for th= em > to be able to establish the tls connection to my syslog server. > > When the clients are unable to verify the server certificate, the server > logs three messages for every connection attempt > > syslog-ng[452597]: SSL error while reading stream; > tls_error=3D'error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unkn= own > ca', location=3D'/etc/syslog-ng/syslog-ng.server.conf:71:17' > syslog-ng[452597]: Error reading RFC6587 style framed data; fd=3D'21', > error=3D'Connection reset by peer (104)' > syslog-ng[452597]: Syslog connection closed; fd=3D'21', > client=3D'AF_INET(1.2.3.4:1234)', local=3D'AF_INET(1.2.3.4:1234)' > > If there are 100's of clients, and they try to reconnect at a fast rate > (every 5 seconds) this can result in a large volume of messages. > > Is there any way to configure the logging rate of these types of errors o= r > get rid of it altogether. > > Anyone have any comments on this? > > > > -- > Evan > > _________________________________________________________________________= _____ > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: > http://www.balabit.com/support/documentation/?product=3Dsyslog-ng > FAQ: http://www.balabit.com/wiki/syslog-ng-faq > > > > -- > Bazsi > > _________________________________________________________________________= _____ > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: > http://www.balabit.com/support/documentation/?product=3Dsyslog-ng > FAQ: http://www.balabit.com/wiki/syslog-ng-faq > > --=20 Bazsi --00000000000055f18c0625028fb0 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div>you can always use rate-limit: <a href=3D"https://axo= flow.com/docs/axosyslog-core/chapter-routing-filters/filters/reference-filt= ers/filter-rate-limit/">https://axoflow.com/docs/axosyslog-core/chapter-rou= ting-filters/filters/reference-filters/filter-rate-limit/</a></div><div><br= ></div><div>Balazs</div><div><br></div></div><br><div class=3D"gmail_quote"= ><div dir=3D"ltr" class=3D"gmail_attr">On Mon, Oct 21, 2024 at 9:58=E2=80= =AFPM Evan Rempel <<a href=3D"mailto:[email protected]">[email protected]</a= >> wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px= 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><di= v class=3D"msg-3622261664690451922"> <div dir=3D"ltr"> <div style=3D"font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Cali= bri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"> I don't want to filter them out completely. This still represents an is= sue and the client is unable to log the issue to the central log/SIEM/monit= oring server. I just don't want to fill up my disks with message repeat= edly. I only need to "hear about it" once every hour (or some other throttled period).</div> <div style=3D"font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Cali= bri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"> <br> </div> <div style=3D"font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Cali= bri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"> Technically there are lots of ways to leverage syslog-ng configurations to = address this but I thought that since this would be an issue for anyone tha= t has a lot of syslog clients and then has an issue with their certificate,= I thought there might be a cleaner and more simple way to address than the correlation/throttling/filtering t= ools of syslog-ng.</div> <div id=3D"m_-3622261664690451922Signature"> <div style=3D"font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Cali= bri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"> <br> </div> <div style=3D"font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Cali= bri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"> For me, the log rate is so large that if I have only a small % of my client= s that have this issue, they will take the service off-line for all of the = other clients.</div> <div style=3D"font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Cali= bri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"> <br> </div> <div style=3D"font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Cali= bri,Helvetica,sans-serif;font-size:12pt;color:rgb(153,153,153)"> --</div> <div style=3D"font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Cali= bri,Helvetica,sans-serif;font-size:12pt;color:rgb(153,153,153)"> Evan</div> </div> <div id=3D"m_-3622261664690451922appendonsend"></div> <hr style=3D"display:inline-block;width:98%"> <div id=3D"m_-3622261664690451922divRplyFwdMsg" dir=3D"ltr"><font face=3D"C= alibri, sans-serif" style=3D"font-size:11pt" color=3D"#000000"><b>From:</b>= syslog-ng <<a href=3D"mailto:[email protected]" target= =3D"_blank">[email protected]</a>> on behalf of Balazs = Scheidler <<a href=3D"mailto:[email protected]" target=3D"_blank">bazsi7= [email protected]</a>><br> <b>Sent:</b> October 21, 2024 12:47 PM<br> <b>To:</b> Syslog-ng users' and developers' mailing list <<a hre= f=3D"mailto:[email protected]" target=3D"_blank">[email protected]= alabit.hu</a>><br> <b>Subject:</b> Re: [syslog-ng] certificate errors result in excessive logg= ing.</font> <div>=C2=A0</div> </div> <div> <table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" width=3D"100%" alig= n=3D"left" style=3D"background:revert;color:revert;direction:revert;font-si= ze:revert;height:revert;letter-spacing:revert;line-height:revert;margin:rev= ert;opacity:revert;outline:revert;overflow:revert;padding:revert;text-align= :revert;text-indent:revert;text-orientation:revert;text-overflow:revert;tex= t-transform:revert;vertical-align:revert;white-space:revert;word-break:reve= rt;word-spacing:revert;writing-mode:revert;zoom:revert;border:0px;display:t= able;width:100%;table-layout:fixed;float:none;border-spacing:0px"> <tbody style=3D"background:revert;border:revert;color:revert;direction:reve= rt;font-size:revert;height:revert;letter-spacing:revert;line-height:revert;= margin:revert;opacity:revert;outline:revert;overflow:revert;padding:revert;= table-layout:revert;text-align:revert;text-indent:revert;text-orientation:r= evert;text-overflow:revert;text-transform:revert;vertical-align:revert;whit= e-space:revert;width:revert;word-break:revert;word-spacing:revert;writing-m= ode:revert;zoom:revert;display:block"> <tr style=3D"background:revert;border:revert;color:revert;direction:revert;= display:revert;font-size:revert;height:revert;letter-spacing:revert;line-he= ight:revert;margin:revert;opacity:revert;outline:revert;overflow:revert;pad= ding:revert;table-layout:revert;text-align:revert;text-indent:revert;text-o= rientation:revert;text-overflow:revert;text-transform:revert;vertical-align= :revert;white-space:revert;width:revert;word-break:revert;word-spacing:reve= rt;writing-mode:revert;zoom:revert"> <td valign=3D"middle" width=3D"1px" bgcolor=3D"#A6A6A6" cellpadding=3D"7px = 2px 7px 2px" style=3D"background-position:revert;background-repeat:revert;b= ackground-image:revert;background-size:revert;background-origin:revert;back= ground-clip:revert;border:revert;color:revert;direction:revert;display:reve= rt;font-size:revert;height:revert;letter-spacing:revert;line-height:revert;= margin:revert;opacity:revert;outline:revert;overflow:revert;table-layout:re= vert;text-align:revert;text-indent:revert;text-orientation:revert;text-over= flow:revert;text-transform:revert;vertical-align:revert;white-space:revert;= word-break:revert;word-spacing:revert;writing-mode:revert;zoom:revert;paddi= ng:7px 2px;background-color:rgb(166,166,166);width:0px"> </td> <td valign=3D"middle" width=3D"100%" bgcolor=3D"#EAEAEA" cellpadding=3D"7px= 5px 7px 15px" color=3D"#212121" style=3D"background-position:revert;backgr= ound-repeat:revert;background-image:revert;background-size:revert;backgroun= d-origin:revert;background-clip:revert;border:revert;direction:revert;displ= ay:revert;height:revert;letter-spacing:revert;line-height:revert;margin:rev= ert;opacity:revert;outline:revert;overflow:revert;table-layout:revert;text-= indent:revert;text-orientation:revert;text-overflow:revert;text-transform:r= evert;vertical-align:revert;white-space:revert;word-break:revert;word-spaci= ng:revert;writing-mode:revert;zoom:revert;width:100%;background-color:rgb(2= 34,234,234);padding:7px 5px 7px 15px;font-family:wf_segoe-ui_normal,Segoe U= I,Segoe WP,Tahoma,Arial,sans-serif;font-size:12px;font-weight:normal;color:= rgb(33,33,33);text-align:left"> <div style=3D"background:revert;border:revert;color:revert;direction:revert= ;display:revert;font-size:revert;height:revert;letter-spacing:revert;line-h= eight:revert;margin:revert;opacity:revert;outline:revert;overflow:revert;pa= dding:revert;table-layout:revert;text-align:revert;text-indent:revert;text-= orientation:revert;text-overflow:revert;text-transform:revert;vertical-alig= n:revert;white-space:revert;width:revert;word-break:revert;word-spacing:rev= ert;writing-mode:revert;zoom:revert"> You don't often get email from <a href=3D"mailto:[email protected]" tar= get=3D"_blank">[email protected]</a>. <a href=3D"https://aka.ms/LearnAboutS= enderIdentification" style=3D"background:revert;color:revert;direction:reve= rt;display:revert;font-size:revert;opacity:revert" target=3D"_blank"> Learn why this is important</a> </div> </td> <td valign=3D"middle" align=3D"left" width=3D"75px" bgcolor=3D"#EAEAEA" cel= lpadding=3D"7px 5px 7px 5px" color=3D"#212121" style=3D"background-position= :revert;background-repeat:revert;background-image:revert;background-size:re= vert;background-origin:revert;background-clip:revert;border:revert;directio= n:revert;display:revert;height:revert;letter-spacing:revert;line-height:rev= ert;margin:revert;opacity:revert;outline:revert;overflow:revert;table-layou= t:revert;text-indent:revert;text-orientation:revert;text-overflow:revert;te= xt-transform:revert;vertical-align:revert;white-space:revert;word-break:rev= ert;word-spacing:revert;writing-mode:revert;zoom:revert;width:75px;backgrou= nd-color:rgb(234,234,234);padding:7px 5px;font-family:wf_segoe-ui_normal,Se= goe UI,Segoe WP,Tahoma,Arial,sans-serif;font-size:12px;font-weight:normal;c= olor:rgb(33,33,33);text-align:left"> </td> </tr> </tbody> </table> <div> <div dir=3D"ltr"> <div>hi,</div> <div><br> </div> <div>I literally submitted a patch this morning to mute #3 from the list ab= ove.</div> <div><a href=3D"https://github.com/axoflow/axosyslog/pull/352" target=3D"_b= lank">https://github.com/axoflow/axosyslog/pull/352</a></div> <div><br> </div> <div>As for the other two, you can filter these out from your internal() so= urce using regexps.</div> <div>Bazsi<br> </div> </div> <br> <div> <div dir=3D"ltr">On Mon, Oct 21, 2024 at 9:30=E2=80=AFPM Evan Rempel <<a= href=3D"mailto:[email protected]" target=3D"_blank">[email protected]</a>> = wrote:<br> </div> <blockquote style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204= ,204,204);padding-left:1ex"> <div> <div dir=3D"ltr"> <div style=3D"font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Cali= bri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"> I am using tls configuration with a locally signed certificate. This mans t= hat I have to configure a custom root CA on to all client systems for them = to be able to establish the tls connection to my syslog server.</div> <div style=3D"font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Cali= bri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"> <br> </div> <div style=3D"font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Cali= bri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"> When the clients are unable to verify the server certificate, the server lo= gs three messages for every connection attempt</div> <div style=3D"font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Cali= bri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"> <br> </div> <div style=3D"font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Cali= bri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"> syslog-ng[452597]: SSL error while reading stream; tls_error=3D'error:1= 4094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca', location= =3D'/etc/syslog-ng/syslog-ng.server.conf:71:17'</div> <div style=3D"font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Cali= bri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"> syslog-ng[452597]: Error reading RFC6587 style framed data; fd=3D'21= 9;, error=3D'Connection reset by peer (104)'</div> <div style=3D"font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Cali= bri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"> syslog-ng[452597]: Syslog connection closed; fd=3D'21', client=3D&#= 39;AF_INET(1.2.3.4:1234)', local=3D'AF_INET(<a href=3D"http://1.2.3= .4:1234/" target=3D"_blank">1.2.3.4:1234</a>)'</div> <div style=3D"font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Cali= bri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"> <br> </div> <div style=3D"font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Cali= bri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"> If there are 100's of clients, and they try to reconnect at a fast rate= (every 5 seconds) this can result in a large volume of messages.</div> <div style=3D"font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Cali= bri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"> <br> </div> <div style=3D"font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Cali= bri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"> Is there any way to configure the logging rate of these types of errors or = get rid of it altogether.</div> <div style=3D"font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Cali= bri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"> <br> </div> <div style=3D"font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Cali= bri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"> Anyone have any comments on this?</div> <div style=3D"font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Cali= bri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"> <br> </div> <div style=3D"font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Cali= bri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"> <br> </div> <div id=3D"m_-3622261664690451922x_m_4590914075091254945Signature"> <div style=3D"font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Cali= bri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"> <br> </div> <div style=3D"font-family:"Courier New",monospace;font-size:14.66= 67px;color:rgb(153,153,153)"> <span style=3D"background-color:rgb(255,255,255)">--</span></div> <div style=3D"font-family:"Courier New",monospace;font-size:14.66= 67px;color:rgb(153,153,153)"> <span style=3D"background-color:rgb(255,255,255)">Evan</span></div> </div> </div> ___________________________________________________________________________= ___<br> Member info: <a href=3D"https://lists.balabit.hu/mailman/listinfo/syslog-ng= " rel=3D"noreferrer" target=3D"_blank"> https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br> Documentation: <a href=3D"http://www.balabit.com/support/documentation/?pro= duct=3Dsyslog-ng" rel=3D"noreferrer" target=3D"_blank"> http://www.balabit.com/support/documentation/?product=3Dsyslog-ng</a><br> FAQ: <a href=3D"http://www.balabit.com/wiki/syslog-ng-faq" rel=3D"noreferre= r" target=3D"_blank"> http://www.balabit.com/wiki/syslog-ng-faq</a><br> <br> </div> </blockquote> </div> <br clear=3D"all"> <br> <span>-- </span><br> <div dir=3D"ltr">Bazsi</div> </div> </div> </div> ___________________________________________________________________________= ___<br> Member info: <a href=3D"https://lists.balabit.hu/mailman/listinfo/syslog-ng= " rel=3D"noreferrer" target=3D"_blank">https://lists.balabit.hu/mailman/lis= tinfo/syslog-ng</a><br> Documentation: <a href=3D"http://www.balabit.com/support/documentation/?pro= duct=3Dsyslog-ng" rel=3D"noreferrer" target=3D"_blank">http://www.balabit.c= om/support/documentation/?product=3Dsyslog-ng</a><br> FAQ: <a href=3D"http://www.balabit.com/wiki/syslog-ng-faq" rel=3D"noreferre= r" target=3D"_blank">http://www.balabit.com/wiki/syslog-ng-faq</a><br> <br> </div></blockquote></div><br clear=3D"all"><br><span class=3D"gmail_signatu= re_prefix">-- </span><br><div dir=3D"ltr" class=3D"gmail_signature">Bazsi</= div> --00000000000055f18c0625028fb0-- --===============8937749272506296314== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq --===============8937749272506296314==--