Re: SA is inventing URIs
Alex <[email protected]> Wed, 24 Jun 2026 22:22:01 -0400
| Newsgroups | gmane.mail.spam.spamassassin.general |
|---|---|
| Message-ID | <CAB1R3sgy_trevy3q_xDJmTBM-Nca1V68qXgArmsAe4iVPQRCKw@mail.gmail.com> |
--000000000000f3da7406550aa89e Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi, I hoped I could follow up on this. The issue is most certainly with the redirectors plugin from v402. # ls -l /usr/share/perl5/vendor_perl/Mail/SpamAssassin/Plugin/Redirectors.p= m -r--r--r-- 1 root root 43525 Jun 21 20:00 /usr/share/perl5/vendor_perl/Mail/SpamAssassin/Plugin/Redirectors.pm Jun 24 22:20:58.866 [1769470] dbg: uri: canonicalizing html uri: https://substack.com/signup?utm_source=3Dsubstack&utm_medium=3Demail&utm_co= ntent=3Dfooter&utm_campaign=3Dautofilled-footer&freeSignupEmail=3Dbulwark@e= xample.com&r=3Dto8ex Jun 24 22:20:58.866 [1769470] dbg: uri: cleaned uri: https://substack.com/signup?utm_source=3Dsubstack&utm_medium=3Demail&utm_co= ntent=3Dfooter&utm_campaign=3Dautofilled-footer&freeSignupEmail=3Dbulwark@e= xample.com&r=3Dto8ex Jun 24 22:20:58.948 [1769470] dbg: Redirectors: Found embedded uri to8ex in https://substack.com/signup?utm_source=3Dsubstack&utm_medium=3Demail&utm_co= ntent=3Dfooter&utm_campaign=3Dautofilled-footer&freeSignupEmail=3Dbulwark@e= xample.com&r=3Dto8ex Jun 24 22:20:58.948 [1769470] dbg: uri: canonicalizing parsed uri: http://to8ex Jun 24 22:20:58.948 [1769470] dbg: uri: cleaned uri: http://www.to8ex.com Jun 24 22:20:58.948 [1769470] dbg: uri: added host: www.to8ex.com domain: to8ex.com Jun 24 22:20:58.949 [1769470] dbg: uri: cleaned uri: http://to8ex Thanks, Alex On Sun, Jun 21, 2026 at 12:59=E2=80=AFPM Alex <[email protected]> wrot= e: > Hi, > I tried to trace this further, and I believe it's related to the > Redirectors plugin from v402. > > When I comment this line in v402.pre: > > loadplugin Mail::SpamAssassin::Plugin::Redirectors > > SpamAssassin no longer extracts or checks to8ex / to8ex.com from: > > https://substack.com/signup?r=3Dto8ex > > With the plugin enabled, the debug output shows: > > dbg: uri: canonicalizing html uri: > https://substack.com/signup?utm_source=3Dsubstack&utm_medium=3Demail&utm_= content=3Dfooter&utm_campaign=3Dautofilled-footer&freeSignupEmail=3Dbulwark= @example.com&r=3Dto8ex > dbg: uri: cleaned uri: > https://substack.com/signup?utm_source=3Dsubstack&utm_medium=3Demail&utm_= content=3Dfooter&utm_campaign=3Dautofilled-footer&freeSignupEmail=3Dbulwark= @example.com&r=3Dto8ex > dbg: uri: canonicalizing parsed uri: http://to8ex > dbg: uri: cleaned uri: http://www.to8ex.com > dbg: uri: added host: www.to8ex.com domain: to8ex.com > dbg: uri: cleaned uri: http://to8ex > dbg: uridnsbl: considering host=3Dwww.to8ex.com, domain=3Dto8ex.com > > With Redirectors disabled, the synthetic to8ex.com URI is not generated. > > I do not have a local redirector_pattern or url_redirector entry for > substack.com/signup. The only stock rule I found appears to be: > > url_redirector substack.com/redirect/ > > So the question is whether Redirectors is treating arbitrary short query > parameter values, such as r=3Dto8ex, as redirect targets and canonicalizi= ng > them as hostnames. That seems unsafe because it causes URIDNSBL checks > against domains that were not actually present in the message. > > On Sat, Jun 20, 2026 at 10:49=E2=80=AFPM John Hardin <[email protected]>= wrote: > >> On Sat, 20 Jun 2026, Alex wrote: >> >> > SpamAssassin 4.0.3 appears to incorrectly promote a URL query >> > parameter value into a standalone URI hostname and then performs >> > URIDNSBL lookups against the generated domain. >> > >> > Observed behavior: >> > >> > Input message contains only the following URI: >> > >> > https://substack.com/signup?r=3Dto8ex >> > >> > Debug output shows SpamAssassin correctly parsing the original URI: >> > >> > Jun 20 17:04:48.788 dbg: uri: canonicalizing parsed uri: >> > https://substack.com/signup?r=3Dto8ex >> > Jun 20 17:04:48.788 dbg: uri: cleaned uri: >> https://substack.com/signup?r=3Dto8ex >> > Jun 20 17:04:48.788 dbg: uri: added host: substack.com domain: >> substack.com >> > >> > Immediately afterward, SpamAssassin creates a second URI which does >> > not exist in the message: >> > >> > Jun 20 17:04:48.821 dbg: uri: canonicalizing parsed uri: http://to8ex >> > Jun 20 17:04:48.822 dbg: uri: cleaned uri: http://to8ex >> > Jun 20 17:04:48.822 dbg: uri: cleaned uri: http://www.to8ex.com >> > Jun 20 17:04:48.822 dbg: uri: added host: www.to8ex.com domain: >> to8ex.com >> >> I cannot reproduce this in my sandbox: >> >> jhardin@davinci ~/develop/spamassassin/testing $ grep uri: result >> Jun 20 19:42:00.009 [3067949] dbg: uri: canonicalizing parsed uri: >> https://substack.com/signup?r=3Dto8ex >> Jun 20 19:42:00.009 [3067949] dbg: uri: cleaned uri: >> https://substack.com/signup?r=3Dto8ex >> Jun 20 19:42:00.009 [3067949] dbg: uri: added host: substack.com domain: >> substack.com >> Jun 20 19:42:02.035 [3067949] dbg: uri: running uri_detail >> __URI_DOTCN_SPOOF: https://substack.com/signup?r=3Dto8ex >> Jun 20 19:42:02.039 [3067949] dbg: uri: running uri_detail >> T_MXG_BING_REDIR_SUSP: https://substack.com/signup?r=3Dto8ex >> Jun 20 19:42:02.043 [3067949] dbg: uri: running uri_detail >> __ALL_URIDETAIL_TEXT: https://substack.com/signup?r=3Dto8ex >> Jun 20 19:42:02.044 [3067949] dbg: uri: running uri_detail >> __MXG_UNSUB_LINK01: https://substack.com/signup?r=3Dto8ex >> Jun 20 19:42:02.051 [3067949] dbg: uri: running uri_detail >> MXG_EMAIL_FRAG: https://substack.com/signup?r=3Dto8ex >> Jun 20 19:42:02.201 [3067949] dbg: uri: canonicalizing parsed uri: >> https://substack.com/signup?r=3Dto8ex >> Jun 20 19:42:02.201 [3067949] dbg: uri: cleaned uri: >> https://substack.com/signup?r=3Dto8ex >> Jun 20 19:42:02.201 [3067949] dbg: uri: added host: substack.com domain: >> substack.com >> jhardin@davinci ~/develop/spamassassin/testing $ >> >> >> Do you have a locally-defined redirector rule? There is one for substack >> in the base ruleset but it does not match that pattern: >> >> jhardin@davinci ~/develop/spamassassin/svn/trunk $ grep -r substack >> rules* >> rules/25_url_redirectors.cf: url_redirector substack.com/redirect/ >> jhardin@davinci ~/develop/spamassassin/svn/trunk $ >> >> There are no `redirector_pattern` rules for substack in the base ruleset= . >> >> >> -- >> John Hardin KA7OHZ http://www.impsec.org/~jhardin/ >> [email protected] pgpk -a [email protected] >> key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 >> ----------------------------------------------------------------------- >> 614 days since SpaceX caught the SuperHeavy booster on the first try >> > --000000000000f3da7406550aa89e Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr">Hi, I hoped I could follow up on this.<div><br></div><div>= The issue is most certainly with the redirectors plugin from v402.</div><di= v># ls -l /usr/share/perl5/vendor_perl/Mail/SpamAssassin/Plugin/Redirectors= .pm<br>-r--r--r-- 1 root root 43525 Jun 21 20:00 /usr/share/perl5/vendor_pe= rl/Mail/SpamAssassin/Plugin/Redirectors.pm<br></div><div><br></div><div>Jun= 24 22:20:58.866 [1769470] dbg: uri: canonicalizing html uri: <a href=3D"ht= tps://substack.com/signup?utm_source=3Dsubstack&utm_medium=3Demail&= utm_content=3Dfooter&utm_campaign=3Dautofilled-footer&freeSignupEma= [email protected]&r=3Dto8ex">https://substack.com/signup?utm_sou= rce=3Dsubstack&utm_medium=3Demail&utm_content=3Dfooter&utm_camp= aign=3Dautofilled-footer&[email protected]&r=3D= to8ex</a><br>Jun 24 22:20:58.866 [1769470] dbg: uri: cleaned uri: <a href= =3D"https://substack.com/signup?utm_source=3Dsubstack&utm_medium=3Demai= l&utm_content=3Dfooter&utm_campaign=3Dautofilled-footer&freeSig= [email protected]&r=3Dto8ex">https://substack.com/signup?u= tm_source=3Dsubstack&utm_medium=3Demail&utm_content=3Dfooter&ut= m_campaign=3Dautofilled-footer&[email protected]&am= p;r=3Dto8ex</a><br>Jun 24 22:20:58.948 [1769470] dbg: Redirectors: Found em= bedded uri to8ex in <a href=3D"https://substack.com/signup?utm_source=3Dsub= stack&utm_medium=3Demail&utm_content=3Dfooter&utm_campaign=3Dau= tofilled-footer&[email protected]&r=3Dto8ex">ht= tps://substack.com/signup?utm_source=3Dsubstack&utm_medium=3Demail&= utm_content=3Dfooter&utm_campaign=3Dautofilled-footer&freeSignupEma= [email protected]&r=3Dto8ex</a><br>Jun 24 22:20:58.948 [1769470]= dbg: uri: canonicalizing parsed uri: <a href=3D"http://to8ex">http://to8ex= </a><br>Jun 24 22:20:58.948 [1769470] dbg: uri: cleaned uri: <a href=3D"htt= p://www.to8ex.com">http://www.to8ex.com</a><br>Jun 24 22:20:58.948 [1769470= ] dbg: uri: added host: <a href=3D"http://www.to8ex.com">www.to8ex.com</a> = domain: <a href=3D"http://to8ex.com">to8ex.com</a><br>Jun 24 22:20:58.949 [= 1769470] dbg: uri: cleaned uri: <a href=3D"http://to8ex">http://to8ex</a><b= r></div><div><br></div><div>Thanks,<br>Alex</div></div><br><div class=3D"gm= ail_quote gmail_quote_container"><div dir=3D"ltr" class=3D"gmail_attr">On S= un, Jun 21, 2026 at 12:59=E2=80=AFPM Alex <<a href=3D"mailto:mysqlstuden= [email protected]">[email protected]</a>> wrote:<br></div><blockquote cla= ss=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid = rgb(204,204,204);padding-left:1ex"><div dir=3D"ltr">Hi,<div>I tried to trac= e this further, and I believe it's related to the Redirectors plugin fr= om v402.</div><div><br></div><div>When I comment this line in v402.pre:<br>= <br>loadplugin Mail::SpamAssassin::Plugin::Redirectors<br><br>SpamAssassin = no longer extracts or checks to8ex / <a href=3D"http://to8ex.com" target=3D= "_blank">to8ex.com</a> from:<br><br><a href=3D"https://substack.com/signup?= r=3Dto8ex" target=3D"_blank">https://substack.com/signup?r=3Dto8ex</a><br><= br>With the plugin enabled, the debug output shows:</div><div><br></div><di= v>dbg: uri: canonicalizing html uri: <a href=3D"https://substack.com/signup= ?utm_source=3Dsubstack&utm_medium=3Demail&utm_content=3Dfooter&= utm_campaign=3Dautofilled-footer&[email protected]&= amp;r=3Dto8ex" target=3D"_blank">https://substack.com/signup?utm_source=3Ds= ubstack&utm_medium=3Demail&utm_content=3Dfooter&utm_campaign=3D= autofilled-footer&[email protected]&r=3Dto8ex</= a><br>dbg: uri: cleaned uri: <a href=3D"https://substack.com/signup?utm_sou= rce=3Dsubstack&utm_medium=3Demail&utm_content=3Dfooter&utm_camp= aign=3Dautofilled-footer&[email protected]&r=3D= to8ex" target=3D"_blank">https://substack.com/signup?utm_source=3Dsubstack&= amp;utm_medium=3Demail&utm_content=3Dfooter&utm_campaign=3Dautofill= ed-footer&[email protected]&r=3Dto8ex</a><br>db= g: uri: canonicalizing parsed uri: <a href=3D"http://to8ex" target=3D"_blan= k">http://to8ex</a><br>dbg: uri: cleaned uri: <a href=3D"http://www.to8ex.c= om" target=3D"_blank">http://www.to8ex.com</a><br>dbg: uri: added host: <a = href=3D"http://www.to8ex.com" target=3D"_blank">www.to8ex.com</a> domain: <= a href=3D"http://to8ex.com" target=3D"_blank">to8ex.com</a><br>dbg: uri: cl= eaned uri: <a href=3D"http://to8ex" target=3D"_blank">http://to8ex</a><br>d= bg: uridnsbl: considering host=3D<a href=3D"http://www.to8ex.com" target=3D= "_blank">www.to8ex.com</a>, domain=3D<a href=3D"http://to8ex.com" target=3D= "_blank">to8ex.com</a><br></div><div><br></div><div>With Redirectors disabl= ed, the synthetic <a href=3D"http://to8ex.com" target=3D"_blank">to8ex.com<= /a> URI is not generated.<br><br>I do not have a local redirector_pattern o= r url_redirector entry for <a href=3D"http://substack.com/signup" target=3D= "_blank">substack.com/signup</a>. The only stock rule I found appears to be= :<br><br>url_redirector <a href=3D"http://substack.com/redirect/" target=3D= "_blank">substack.com/redirect/</a><br><br>So the question is whether Redir= ectors is treating arbitrary short query parameter values, such as r=3Dto8e= x, as redirect targets and canonicalizing them as hostnames. That seems uns= afe because it causes URIDNSBL checks against domains that were not actuall= y present in the message.</div></div><br><div class=3D"gmail_quote"><div di= r=3D"ltr" class=3D"gmail_attr">On Sat, Jun 20, 2026 at 10:49=E2=80=AFPM Joh= n Hardin <<a href=3D"mailto:[email protected]" target=3D"_blank">jhardi= [email protected]</a>> wrote:<br></div><blockquote class=3D"gmail_quote" styl= e=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);paddin= g-left:1ex">On Sat, 20 Jun 2026, Alex wrote:<br> <br> > SpamAssassin 4.0.3 appears to incorrectly promote a URL query<br> > parameter value into a standalone URI hostname and then performs<br> > URIDNSBL lookups against the generated domain.<br> ><br> > Observed behavior:<br> ><br> > Input message contains only the following URI:<br> ><br> > <a href=3D"https://substack.com/signup?r=3Dto8ex" rel=3D"noreferrer" t= arget=3D"_blank">https://substack.com/signup?r=3Dto8ex</a><br> ><br> > Debug output shows SpamAssassin correctly parsing the original URI:<br= > ><br> > Jun 20 17:04:48.788 dbg: uri: canonicalizing parsed uri:<br> > <a href=3D"https://substack.com/signup?r=3Dto8ex" rel=3D"noreferrer" t= arget=3D"_blank">https://substack.com/signup?r=3Dto8ex</a><br> > Jun 20 17:04:48.788 dbg: uri: cleaned uri: <a href=3D"https://substack= .com/signup?r=3Dto8ex" rel=3D"noreferrer" target=3D"_blank">https://substac= k.com/signup?r=3Dto8ex</a><br> > Jun 20 17:04:48.788 dbg: uri: added host: <a href=3D"http://substack.c= om" rel=3D"noreferrer" target=3D"_blank">substack.com</a> domain: <a href= =3D"http://substack.com" rel=3D"noreferrer" target=3D"_blank">substack.com<= /a><br> ><br> > Immediately afterward, SpamAssassin creates a second URI which does<br= > > not exist in the message:<br> ><br> > Jun 20 17:04:48.821 dbg: uri: canonicalizing parsed uri: <a href=3D"ht= tp://to8ex" rel=3D"noreferrer" target=3D"_blank">http://to8ex</a><br> > Jun 20 17:04:48.822 dbg: uri: cleaned uri: <a href=3D"http://to8ex" re= l=3D"noreferrer" target=3D"_blank">http://to8ex</a><br> > Jun 20 17:04:48.822 dbg: uri: cleaned uri: <a href=3D"http://www.to8ex= .com" rel=3D"noreferrer" target=3D"_blank">http://www.to8ex.com</a><br> > Jun 20 17:04:48.822 dbg: uri: added host: <a href=3D"http://www.to8ex.= com" rel=3D"noreferrer" target=3D"_blank">www.to8ex.com</a> domain: <a href= =3D"http://to8ex.com" rel=3D"noreferrer" target=3D"_blank">to8ex.com</a><br= > <br> I cannot reproduce this in my sandbox:<br> <br> jhardin@davinci ~/develop/spamassassin/testing $ grep uri: result<br> Jun 20 19:42:00.009 [3067949] dbg: uri: canonicalizing parsed uri: <a href= =3D"https://substack.com/signup?r=3Dto8ex" rel=3D"noreferrer" target=3D"_bl= ank">https://substack.com/signup?r=3Dto8ex</a><br> Jun 20 19:42:00.009 [3067949] dbg: uri: cleaned uri: <a href=3D"https://sub= stack.com/signup?r=3Dto8ex" rel=3D"noreferrer" target=3D"_blank">https://su= bstack.com/signup?r=3Dto8ex</a><br> Jun 20 19:42:00.009 [3067949] dbg: uri: added host: <a href=3D"http://subst= ack.com" rel=3D"noreferrer" target=3D"_blank">substack.com</a> domain: <a h= ref=3D"http://substack.com" rel=3D"noreferrer" target=3D"_blank">substack.c= om</a><br> Jun 20 19:42:02.035 [3067949] dbg: uri: running uri_detail __URI_DOTCN_SPOO= F: <a href=3D"https://substack.com/signup?r=3Dto8ex" rel=3D"noreferrer" tar= get=3D"_blank">https://substack.com/signup?r=3Dto8ex</a><br> Jun 20 19:42:02.039 [3067949] dbg: uri: running uri_detail T_MXG_BING_REDIR= _SUSP: <a href=3D"https://substack.com/signup?r=3Dto8ex" rel=3D"noreferrer"= target=3D"_blank">https://substack.com/signup?r=3Dto8ex</a><br> Jun 20 19:42:02.043 [3067949] dbg: uri: running uri_detail __ALL_URIDETAIL_= TEXT: <a href=3D"https://substack.com/signup?r=3Dto8ex" rel=3D"noreferrer" = target=3D"_blank">https://substack.com/signup?r=3Dto8ex</a><br> Jun 20 19:42:02.044 [3067949] dbg: uri: running uri_detail __MXG_UNSUB_LINK= 01: <a href=3D"https://substack.com/signup?r=3Dto8ex" rel=3D"noreferrer" ta= rget=3D"_blank">https://substack.com/signup?r=3Dto8ex</a><br> Jun 20 19:42:02.051 [3067949] dbg: uri: running uri_detail MXG_EMAIL_FRAG: = <a href=3D"https://substack.com/signup?r=3Dto8ex" rel=3D"noreferrer" target= =3D"_blank">https://substack.com/signup?r=3Dto8ex</a><br> Jun 20 19:42:02.201 [3067949] dbg: uri: canonicalizing parsed uri: <a href= =3D"https://substack.com/signup?r=3Dto8ex" rel=3D"noreferrer" target=3D"_bl= ank">https://substack.com/signup?r=3Dto8ex</a><br> Jun 20 19:42:02.201 [3067949] dbg: uri: cleaned uri: <a href=3D"https://sub= stack.com/signup?r=3Dto8ex" rel=3D"noreferrer" target=3D"_blank">https://su= bstack.com/signup?r=3Dto8ex</a><br> Jun 20 19:42:02.201 [3067949] dbg: uri: added host: <a href=3D"http://subst= ack.com" rel=3D"noreferrer" target=3D"_blank">substack.com</a> domain: <a h= ref=3D"http://substack.com" rel=3D"noreferrer" target=3D"_blank">substack.c= om</a><br> jhardin@davinci ~/develop/spamassassin/testing $<br> <br> <br> Do you have a locally-defined redirector rule? There is one for substack <b= r> in the base ruleset but it does not match that pattern:<br> <br> jhardin@davinci ~/develop/spamassassin/svn/trunk $ grep -r substack rules*<= br> rules/<a href=3D"http://25_url_redirectors.cf" rel=3D"noreferrer" target=3D= "_blank">25_url_redirectors.cf</a>:=C2=A0 =C2=A0 url_redirector <a href=3D"= http://substack.com/redirect/" rel=3D"noreferrer" target=3D"_blank">substac= k.com/redirect/</a><br> jhardin@davinci ~/develop/spamassassin/svn/trunk $<br> <br> There are no `redirector_pattern` rules for substack in the base ruleset.<b= r> <br> <br> -- <br> =C2=A0 John Hardin KA7OHZ=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 <a href=3D"http://www.impsec.org/~jhardin/" rel=3D"nor= eferrer" target=3D"_blank">http://www.impsec.org/~jhardin/</a><br> =C2=A0 <a href=3D"mailto:[email protected]" target=3D"_blank">jhardin@imps= ec.org</a>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0pgpk -a <a href=3D"mailto:[email protected]" ta= rget=3D"_blank">[email protected]</a><br> =C2=A0 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C=C2=A0 AF76 D822 E6E6 B87= 3 2E79<br> -----------------------------------------------------------------------<br> =C2=A0 614 days since SpaceX caught the SuperHeavy booster on the first try= <br> </blockquote></div> </blockquote></div> --000000000000f3da7406550aa89e--