Re: SA is inventing URIs
Alex <[email protected]> Sun, 21 Jun 2026 12:59:24 -0400
| Newsgroups | gmane.mail.spam.spamassassin.general |
|---|---|
| Message-ID | <CAB1R3sg7_UHZcu_QYHjjZh+cKqpTif8+SzVU3qp8U1nNCA2k0Q@mail.gmail.com> |
--00000000000065d9200654c673a2 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi, I tried to trace this further, and I believe it's related to the Redirectors plugin from v402. When I comment this line in v402.pre: loadplugin Mail::SpamAssassin::Plugin::Redirectors SpamAssassin no longer extracts or checks to8ex / to8ex.com from: https://substack.com/signup?r=3Dto8ex With the plugin enabled, the debug output shows: dbg: uri: canonicalizing html uri: https://substack.com/signup?utm_source=3Dsubstack&utm_medium=3Demail&utm_co= ntent=3Dfooter&utm_campaign=3Dautofilled-footer&freeSignupEmail=3Dbulwark@e= xample.com&r=3Dto8ex dbg: uri: cleaned uri: https://substack.com/signup?utm_source=3Dsubstack&utm_medium=3Demail&utm_co= ntent=3Dfooter&utm_campaign=3Dautofilled-footer&freeSignupEmail=3Dbulwark@e= xample.com&r=3Dto8ex dbg: uri: canonicalizing parsed uri: http://to8ex dbg: uri: cleaned uri: http://www.to8ex.com dbg: uri: added host: www.to8ex.com domain: to8ex.com dbg: uri: cleaned uri: http://to8ex dbg: uridnsbl: considering host=3Dwww.to8ex.com, domain=3Dto8ex.com With Redirectors disabled, the synthetic to8ex.com URI is not generated. I do not have a local redirector_pattern or url_redirector entry for substack.com/signup. The only stock rule I found appears to be: url_redirector substack.com/redirect/ So the question is whether Redirectors is treating arbitrary short query parameter values, such as r=3Dto8ex, as redirect targets and canonicalizing them as hostnames. That seems unsafe because it causes URIDNSBL checks against domains that were not actually present in the message. On Sat, Jun 20, 2026 at 10:49=E2=80=AFPM John Hardin <[email protected]> w= rote: > On Sat, 20 Jun 2026, Alex wrote: > > > SpamAssassin 4.0.3 appears to incorrectly promote a URL query > > parameter value into a standalone URI hostname and then performs > > URIDNSBL lookups against the generated domain. > > > > Observed behavior: > > > > Input message contains only the following URI: > > > > https://substack.com/signup?r=3Dto8ex > > > > Debug output shows SpamAssassin correctly parsing the original URI: > > > > Jun 20 17:04:48.788 dbg: uri: canonicalizing parsed uri: > > https://substack.com/signup?r=3Dto8ex > > Jun 20 17:04:48.788 dbg: uri: cleaned uri: > https://substack.com/signup?r=3Dto8ex > > Jun 20 17:04:48.788 dbg: uri: added host: substack.com domain: > substack.com > > > > Immediately afterward, SpamAssassin creates a second URI which does > > not exist in the message: > > > > Jun 20 17:04:48.821 dbg: uri: canonicalizing parsed uri: http://to8ex > > Jun 20 17:04:48.822 dbg: uri: cleaned uri: http://to8ex > > Jun 20 17:04:48.822 dbg: uri: cleaned uri: http://www.to8ex.com > > Jun 20 17:04:48.822 dbg: uri: added host: www.to8ex.com domain: > to8ex.com > > I cannot reproduce this in my sandbox: > > jhardin@davinci ~/develop/spamassassin/testing $ grep uri: result > Jun 20 19:42:00.009 [3067949] dbg: uri: canonicalizing parsed uri: > https://substack.com/signup?r=3Dto8ex > Jun 20 19:42:00.009 [3067949] dbg: uri: cleaned uri: > https://substack.com/signup?r=3Dto8ex > Jun 20 19:42:00.009 [3067949] dbg: uri: added host: substack.com domain: > substack.com > Jun 20 19:42:02.035 [3067949] dbg: uri: running uri_detail > __URI_DOTCN_SPOOF: https://substack.com/signup?r=3Dto8ex > Jun 20 19:42:02.039 [3067949] dbg: uri: running uri_detail > T_MXG_BING_REDIR_SUSP: https://substack.com/signup?r=3Dto8ex > Jun 20 19:42:02.043 [3067949] dbg: uri: running uri_detail > __ALL_URIDETAIL_TEXT: https://substack.com/signup?r=3Dto8ex > Jun 20 19:42:02.044 [3067949] dbg: uri: running uri_detail > __MXG_UNSUB_LINK01: https://substack.com/signup?r=3Dto8ex > Jun 20 19:42:02.051 [3067949] dbg: uri: running uri_detail MXG_EMAIL_FRAG= : > https://substack.com/signup?r=3Dto8ex > Jun 20 19:42:02.201 [3067949] dbg: uri: canonicalizing parsed uri: > https://substack.com/signup?r=3Dto8ex > Jun 20 19:42:02.201 [3067949] dbg: uri: cleaned uri: > https://substack.com/signup?r=3Dto8ex > Jun 20 19:42:02.201 [3067949] dbg: uri: added host: substack.com domain: > substack.com > jhardin@davinci ~/develop/spamassassin/testing $ > > > Do you have a locally-defined redirector rule? There is one for substack > in the base ruleset but it does not match that pattern: > > jhardin@davinci ~/develop/spamassassin/svn/trunk $ grep -r substack rules= * > rules/25_url_redirectors.cf: url_redirector substack.com/redirect/ > jhardin@davinci ~/develop/spamassassin/svn/trunk $ > > There are no `redirector_pattern` rules for substack in the base ruleset. > > > -- > John Hardin KA7OHZ http://www.impsec.org/~jhardin/ > [email protected] pgpk -a [email protected] > key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 > ----------------------------------------------------------------------- > 614 days since SpaceX caught the SuperHeavy booster on the first try > --00000000000065d9200654c673a2 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr">Hi,<div>I tried to trace this further, and I believe it= 9;s related to the Redirectors plugin from v402.</div><div><br></div><div>W= hen I comment this line in v402.pre:<br><br>loadplugin Mail::SpamAssassin::= Plugin::Redirectors<br><br>SpamAssassin no longer extracts or checks to8ex = / <a href=3D"http://to8ex.com">to8ex.com</a> from:<br><br><a href=3D"https:= //substack.com/signup?r=3Dto8ex">https://substack.com/signup?r=3Dto8ex</a><= br><br>With the plugin enabled, the debug output shows:</div><div><br></div= ><div>dbg: uri: canonicalizing html uri: <a href=3D"https://substack.com/si= gnup?utm_source=3Dsubstack&utm_medium=3Demail&utm_content=3Dfooter&= amp;utm_campaign=3Dautofilled-footer&freeSignupEmail=3Dbulwark@example.= com&r=3Dto8ex">https://substack.com/signup?utm_source=3Dsubstack&ut= m_medium=3Demail&utm_content=3Dfooter&utm_campaign=3Dautofilled-foo= ter&[email protected]&r=3Dto8ex</a><br>dbg: uri= : cleaned uri: <a href=3D"https://substack.com/signup?utm_source=3Dsubstack= &utm_medium=3Demail&utm_content=3Dfooter&utm_campaign=3Dautofil= led-footer&[email protected]&r=3Dto8ex">https:/= /substack.com/signup?utm_source=3Dsubstack&utm_medium=3Demail&utm_c= ontent=3Dfooter&utm_campaign=3Dautofilled-footer&freeSignupEmail=3D= [email protected]&r=3Dto8ex</a><br>dbg: uri: canonicalizing parsed ur= i: <a href=3D"http://to8ex">http://to8ex</a><br>dbg: uri: cleaned uri: <a h= ref=3D"http://www.to8ex.com">http://www.to8ex.com</a><br>dbg: uri: added ho= st: <a href=3D"http://www.to8ex.com">www.to8ex.com</a> domain: <a href=3D"h= ttp://to8ex.com">to8ex.com</a><br>dbg: uri: cleaned uri: <a href=3D"http://= to8ex">http://to8ex</a><br>dbg: uridnsbl: considering host=3D<a href=3D"htt= p://www.to8ex.com">www.to8ex.com</a>, domain=3D<a href=3D"http://to8ex.com"= >to8ex.com</a><br></div><div><br></div><div>With Redirectors disabled, the = synthetic <a href=3D"http://to8ex.com">to8ex.com</a> URI is not generated.<= br><br>I do not have a local redirector_pattern or url_redirector entry for= <a href=3D"http://substack.com/signup">substack.com/signup</a>. The only s= tock rule I found appears to be:<br><br>url_redirector <a href=3D"http://su= bstack.com/redirect/">substack.com/redirect/</a><br><br>So the question is = whether Redirectors is treating arbitrary short query parameter values, suc= h as r=3Dto8ex, as redirect targets and canonicalizing them as hostnames. T= hat seems unsafe because it causes URIDNSBL checks against domains that wer= e not actually present in the message.</div></div><br><div class=3D"gmail_q= uote gmail_quote_container"><div dir=3D"ltr" class=3D"gmail_attr">On Sat, J= un 20, 2026 at 10:49=E2=80=AFPM John Hardin <<a href=3D"mailto:jhardin@i= mpsec.org">[email protected]</a>> wrote:<br></div><blockquote class=3D"= gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(20= 4,204,204);padding-left:1ex">On Sat, 20 Jun 2026, Alex wrote:<br> <br> > SpamAssassin 4.0.3 appears to incorrectly promote a URL query<br> > parameter value into a standalone URI hostname and then performs<br> > URIDNSBL lookups against the generated domain.<br> ><br> > Observed behavior:<br> ><br> > Input message contains only the following URI:<br> ><br> > <a href=3D"https://substack.com/signup?r=3Dto8ex" rel=3D"noreferrer" t= arget=3D"_blank">https://substack.com/signup?r=3Dto8ex</a><br> ><br> > Debug output shows SpamAssassin correctly parsing the original URI:<br= > ><br> > Jun 20 17:04:48.788 dbg: uri: canonicalizing parsed uri:<br> > <a href=3D"https://substack.com/signup?r=3Dto8ex" rel=3D"noreferrer" t= arget=3D"_blank">https://substack.com/signup?r=3Dto8ex</a><br> > Jun 20 17:04:48.788 dbg: uri: cleaned uri: <a href=3D"https://substack= .com/signup?r=3Dto8ex" rel=3D"noreferrer" target=3D"_blank">https://substac= k.com/signup?r=3Dto8ex</a><br> > Jun 20 17:04:48.788 dbg: uri: added host: <a href=3D"http://substack.c= om" rel=3D"noreferrer" target=3D"_blank">substack.com</a> domain: <a href= =3D"http://substack.com" rel=3D"noreferrer" target=3D"_blank">substack.com<= /a><br> ><br> > Immediately afterward, SpamAssassin creates a second URI which does<br= > > not exist in the message:<br> ><br> > Jun 20 17:04:48.821 dbg: uri: canonicalizing parsed uri: <a href=3D"ht= tp://to8ex" rel=3D"noreferrer" target=3D"_blank">http://to8ex</a><br> > Jun 20 17:04:48.822 dbg: uri: cleaned uri: <a href=3D"http://to8ex" re= l=3D"noreferrer" target=3D"_blank">http://to8ex</a><br> > Jun 20 17:04:48.822 dbg: uri: cleaned uri: <a href=3D"http://www.to8ex= .com" rel=3D"noreferrer" target=3D"_blank">http://www.to8ex.com</a><br> > Jun 20 17:04:48.822 dbg: uri: added host: <a href=3D"http://www.to8ex.= com" rel=3D"noreferrer" target=3D"_blank">www.to8ex.com</a> domain: <a href= =3D"http://to8ex.com" rel=3D"noreferrer" target=3D"_blank">to8ex.com</a><br= > <br> I cannot reproduce this in my sandbox:<br> <br> jhardin@davinci ~/develop/spamassassin/testing $ grep uri: result<br> Jun 20 19:42:00.009 [3067949] dbg: uri: canonicalizing parsed uri: <a href= =3D"https://substack.com/signup?r=3Dto8ex" rel=3D"noreferrer" target=3D"_bl= ank">https://substack.com/signup?r=3Dto8ex</a><br> Jun 20 19:42:00.009 [3067949] dbg: uri: cleaned uri: <a href=3D"https://sub= stack.com/signup?r=3Dto8ex" rel=3D"noreferrer" target=3D"_blank">https://su= bstack.com/signup?r=3Dto8ex</a><br> Jun 20 19:42:00.009 [3067949] dbg: uri: added host: <a href=3D"http://subst= ack.com" rel=3D"noreferrer" target=3D"_blank">substack.com</a> domain: <a h= ref=3D"http://substack.com" rel=3D"noreferrer" target=3D"_blank">substack.c= om</a><br> Jun 20 19:42:02.035 [3067949] dbg: uri: running uri_detail __URI_DOTCN_SPOO= F: <a href=3D"https://substack.com/signup?r=3Dto8ex" rel=3D"noreferrer" tar= get=3D"_blank">https://substack.com/signup?r=3Dto8ex</a><br> Jun 20 19:42:02.039 [3067949] dbg: uri: running uri_detail T_MXG_BING_REDIR= _SUSP: <a href=3D"https://substack.com/signup?r=3Dto8ex" rel=3D"noreferrer"= target=3D"_blank">https://substack.com/signup?r=3Dto8ex</a><br> Jun 20 19:42:02.043 [3067949] dbg: uri: running uri_detail __ALL_URIDETAIL_= TEXT: <a href=3D"https://substack.com/signup?r=3Dto8ex" rel=3D"noreferrer" = target=3D"_blank">https://substack.com/signup?r=3Dto8ex</a><br> Jun 20 19:42:02.044 [3067949] dbg: uri: running uri_detail __MXG_UNSUB_LINK= 01: <a href=3D"https://substack.com/signup?r=3Dto8ex" rel=3D"noreferrer" ta= rget=3D"_blank">https://substack.com/signup?r=3Dto8ex</a><br> Jun 20 19:42:02.051 [3067949] dbg: uri: running uri_detail MXG_EMAIL_FRAG: = <a href=3D"https://substack.com/signup?r=3Dto8ex" rel=3D"noreferrer" target= =3D"_blank">https://substack.com/signup?r=3Dto8ex</a><br> Jun 20 19:42:02.201 [3067949] dbg: uri: canonicalizing parsed uri: <a href= =3D"https://substack.com/signup?r=3Dto8ex" rel=3D"noreferrer" target=3D"_bl= ank">https://substack.com/signup?r=3Dto8ex</a><br> Jun 20 19:42:02.201 [3067949] dbg: uri: cleaned uri: <a href=3D"https://sub= stack.com/signup?r=3Dto8ex" rel=3D"noreferrer" target=3D"_blank">https://su= bstack.com/signup?r=3Dto8ex</a><br> Jun 20 19:42:02.201 [3067949] dbg: uri: added host: <a href=3D"http://subst= ack.com" rel=3D"noreferrer" target=3D"_blank">substack.com</a> domain: <a h= ref=3D"http://substack.com" rel=3D"noreferrer" target=3D"_blank">substack.c= om</a><br> jhardin@davinci ~/develop/spamassassin/testing $<br> <br> <br> Do you have a locally-defined redirector rule? There is one for substack <b= r> in the base ruleset but it does not match that pattern:<br> <br> jhardin@davinci ~/develop/spamassassin/svn/trunk $ grep -r substack rules*<= br> rules/<a href=3D"http://25_url_redirectors.cf" rel=3D"noreferrer" target=3D= "_blank">25_url_redirectors.cf</a>:=C2=A0 =C2=A0 url_redirector <a href=3D"= http://substack.com/redirect/" rel=3D"noreferrer" target=3D"_blank">substac= k.com/redirect/</a><br> jhardin@davinci ~/develop/spamassassin/svn/trunk $<br> <br> There are no `redirector_pattern` rules for substack in the base ruleset.<b= r> <br> <br> -- <br> =C2=A0 John Hardin KA7OHZ=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 <a href=3D"http://www.impsec.org/~jhardin/" rel=3D"nor= eferrer" target=3D"_blank">http://www.impsec.org/~jhardin/</a><br> =C2=A0 <a href=3D"mailto:[email protected]" target=3D"_blank">jhardin@imps= ec.org</a>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0pgpk -a <a href=3D"mailto:[email protected]" ta= rget=3D"_blank">[email protected]</a><br> =C2=A0 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C=C2=A0 AF76 D822 E6E6 B87= 3 2E79<br> -----------------------------------------------------------------------<br> =C2=A0 614 days since SpaceX caught the SuperHeavy booster on the first try= <br> </blockquote></div> --00000000000065d9200654c673a2--