Re: Phishing Script and Server Update & Future Plans
"Jerry Benton" <jerry.benton-OT/[email protected]> Wed, 6 Nov 2024 14:38:07 +0000
| Newsgroups | gmane.mail.virus.mailscanner |
|---|---|
| Message-ID | <DS7PR12MB8372DB0DBBA013C24C4CFFE2F4532@DS7PR12MB8372.namprd12.prod.outlook.com> |
--===============5356876700151194788== Content-Language: en-US Content-Type: multipart/alternative; boundary="_000_DS7PR12MB8372DB0DBBA013C24C4CFFE2F4532DS7PR12MB8372namp_" --_000_DS7PR12MB8372DB0DBBA013C24C4CFFE2F4532DS7PR12MB8372namp_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable It was not in gzip format because the update server was handing you a 301 t= hat was not being handled correctly. This will no longer happen. -- Jerry Benton www.mailborder.com<http://www.mailborder.com> +1 843-800-8605 From: MailScanner <mailscanner-bounces+jerry.benton=3Dmailborder.com@lists.= mailscanner.info> on behalf of Bogdan-Stefan Rotariu <[email protected]> Date: Wednesday, November 6, 2024 at 09:36 To: MailScanner Discussion <mailscanner-qhrM8SXbD5JTOyd/[email protected]> Subject: Re: Phishing Script and Server Update & Future Plans Thank you for the recent update. I would like to suggest two changes: 1. Since we have already set the PATH, we can simply use ms-peek without sp= ecifying the full path (/usr/sbin/ms-peek). 2. Regarding the enforcement of HTTPS, we have encountered issues with bad = .gz files. Currently, the only fix is to remove them manually. I propose th= at the script either overwrites these .gz files or removes them before down= loading new ones to prevent this issue. gzip: /usr/mailscanner/etc/phishing.safe.sites.conf.master.gz: not in gzip = format I have added a workaround to our script: if [ -d $CONFIGDIR ]; then cd $CONFIGDIR #delete .gz old gz files if left behind find "$CONFIGDIR" -name "*.gz" -type f -exec rm -f {} + Thanks, -- Bogdan-Stefan Rotariu On 6 Nov 2024, at 14:20, Jerry Benton <jerry.benton-OT/[email protected]> wrote: I have updated the cron script for phishing bad/safe sites to v0.3.1. It is= available here:https://phishing.mailscanner.info<https://phishing.mailscan= ner.info/> The phishing update server has been updated to use https, but will not forc= e that connection and will now accept http. Several previous automatic bad site scrubs were removed for the google doma= in and several subdomains. This means you will need to add them to the phis= hing safe sites custom file to retain links to google domains. Uncompressed versions of the bad/safe sites files are no longer available. = If you have an old script trying to download those files, it will fail. You= will need to use the newer update script or modify any custom scripts to d= ownload the compressed versions and then extract them for use. The phishing bad sites is compiled from Phishtank data. It essentially pull= s the domains from the phishing links and puts them into a flat file for us= e in MailScanner. Future plan: I am looking into creating a mechanism to populate a SQLite database with p= hishing links instead of just domains. I am creating this for use in Mailbo= rder, but will make the database available for MailScanner. In order to be = used in MailScanner someone would have to write the code to use it. (I have= enough to do.) Once I create a base structure of what the database would l= ook like, I will send out an update here with a link to the SQLite database= . Not sure how this would impact scanning speed at this time. It would of c= ourse depend on how big that database gets. The reason why I am looking into this =85 I contacted Openphish about licensing cost to get their database. They want= $10k per server per year for the =93light=94 version. That is just nuts. How you can help =85 I need source data to compile this database. I can get some from Phishtank = and a very small amount from Openphish free sources. I haven=92t put a whol= e lot of cycles into this, but if you have any suggestions, please email me= . -- Jerry Benton www.mailborder.com<http://www.mailborder.com/> +1 843-800-8605 -- MailScanner mailing list mailscanner-qhrM8SXbD5JTOyd/[email protected]<mailto:mailscanner-qhrM8SXbD5JTOyd/[email protected]= o> http://lists.mailscanner.info/mailman/listinfo/mailscanner --_000_DS7PR12MB8372DB0DBBA013C24C4CFFE2F4532DS7PR12MB8372namp_ Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable <html xmlns:o=3D"urn:schemas-microsoft-com:office:office" xmlns:w=3D"urn:sc= hemas-microsoft-com:office:word" xmlns:m=3D"http://schemas.microsoft.com/of= fice/2004/12/omml" xmlns=3D"http://www.w3.org/TR/REC-html40"> <head> <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3DWindows-1= 252"> <meta name=3D"Generator" content=3D"Microsoft Word 15 (filtered medium)"> <style><!-- /* Font Definitions */ @font-face {font-family:Helvetica; panose-1:0 0 0 0 0 0 0 0 0 0;} @font-face {font-family:"Cambria Math"; panose-1:2 4 5 3 5 4 6 3 2 4;} @font-face {font-family:Calibri; panose-1:2 15 5 2 2 2 4 3 2 4;} @font-face {font-family:Aptos; panose-1:2 11 0 4 2 2 2 2 2 4;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {margin:0in; font-size:12.0pt; font-family:"Aptos",sans-serif;} a:link, span.MsoHyperlink {mso-style-priority:99; color:blue; text-decoration:underline;} span.apple-converted-space {mso-style-name:apple-converted-space;} span.EmailStyle19 {mso-style-type:personal-reply; font-family:"Aptos",sans-serif; color:windowtext;} .MsoChpDefault {mso-style-type:export-only; font-size:10.0pt; mso-ligatures:none;} @page WordSection1 {size:8.5in 11.0in; margin:1.0in 1.0in 1.0in 1.0in;} div.WordSection1 {page:WordSection1;} --></style> </head> <body lang=3D"EN-US" link=3D"blue" vlink=3D"purple" style=3D"word-wrap:brea= k-word;line-break:after-white-space"> <div class=3D"WordSection1"> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt">It was not in gzip = format because the update server was handing you a 301 that was not being h= andled correctly. This will no longer happen. <o:p></o:p></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt"><o:p> </o:p></= span></p> <div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif"><o:p> </o:p></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif">-- <o:p></o:p></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif">Jerry Benton<o:p></o:p></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif"><a href=3D"http://www.mailborder.com"><span style= =3D"color:#0563C1">www.mailborder.com</span></a><o:p></o:p></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif">+1 843-800-8605<o:p></o:p></span></p> </div> </div> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt"><o:p> </o:p></= span></p> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt"><o:p> </o:p></= span></p> <div id=3D"mail-editor-reference-message-container"> <div> <div> <div style=3D"border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in = 0in 0in"> <p class=3D"MsoNormal" style=3D"margin-bottom:12.0pt"><b><span style=3D"col= or:black">From: </span></b><span style=3D"color:black">MailScanner <mailscanner-bounces+= jerry.benton=3Dmailborder.com-qhrM8SXbD5JTOyd/[email protected]> on behalf of Bogda= n-Stefan Rotariu <[email protected]><br> <b>Date: </b>Wednesday, November 6, 2024 at 09:36<br> <b>To: </b>MailScanner Discussion <mailscanner-qhrM8SXbD5JTOyd/[email protected]>= ;<br> <b>Subject: </b>Re: Phishing Script and Server Update & Future Plans<o:= p></o:p></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"color:black">Thank you for the recent= update. I would like to suggest two changes:<o:p></o:p></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"color:black"><br> 1. Since we have already set the PATH, we can simply use ms-peek witho= ut specifying the full path (/usr/sbin/ms-peek).<o:p></o:p></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"color:black">2. Regarding the en= forcement of HTTPS, we have encountered issues with bad .gz files. Currentl= y, the only fix is to remove them manually. I propose that the script eithe= r overwrites these .gz files or removes them before downloading new ones to prevent this issue.<o:p></o:p></span></p> </div> <p class=3D"MsoNormal"><span style=3D"color:black"><br> gzip: /usr/mailscanner/etc/phishing.safe.sites.conf.master.gz: not in gzip = format</span><o:p></o:p></p> <div> <p class=3D"MsoNormal"><span style=3D"color:black"><br> I have added a workaround to our script:<o:p></o:p></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"color:black"><o:p> </o:p></span>= </p> </div> <div> <p class=3D"MsoNormal"><span style=3D"color:black">if [ -d $CONFIGDIR ]; th= en<br> cd $CONFIGDIR<br> #delete .gz old gz files if left behind<br> find "$CONFIGDIR" -name "*.gz" -type f -exec rm -f {} += <br> <br> Thanks,<o:p></o:p></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"color:black"><o:p> </o:p></span>= </p> </div> <div> <div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:9.0pt;font-family:Helvetica= ;color:black">--<o:p></o:p></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:9.0pt;font-family:Helvetica= ;color:black">Bogdan-Stefan Rotariu<o:p></o:p></span></p> </div> <p class=3D"MsoNormal"><o:p> </o:p></p> </div> <p class=3D"MsoNormal"><o:p> </o:p></p> </div> <div> <p class=3D"MsoNormal"><br> <br> <o:p></o:p></p> <blockquote style=3D"margin-top:5.0pt;margin-bottom:5.0pt"> <div> <p class=3D"MsoNormal">On 6 Nov 2024, at 14:20, Jerry Benton <jerry.bent= on-OT/[email protected]> wrote:<o:p></o:p></p> </div> <p class=3D"MsoNormal"><o:p> </o:p></p> <div> <div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt">I have updated the = cron script for phishing bad/safe sites to v0.3.1. It is available here:<a = href=3D"https://phishing.mailscanner.info/"><span style=3D"color:#467886">h= ttps://phishing.mailscanner.info</span></a><o:p></o:p></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt"> <o:p></o:p></= span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt">The phishing update= server has been updated to use https, but will not force that connection a= nd will now accept http.<o:p></o:p></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt"> <o:p></o:p></= span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt">Several previous au= tomatic bad site scrubs were removed for the google domain and several subd= omains. This means you will need to add them to the phishing safe sites cus= tom file to retain links to google domains.<o:p></o:p></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt"> <o:p></o:p></= span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt">Uncompressed versio= ns of the bad/safe sites files are no longer available. If you have an old = script trying to download those files, it will fail. You will need to use t= he newer update script or modify any custom scripts to download the compressed versions and then extract them f= or use.<span class=3D"apple-converted-space"> </span><o:p></o:p></span= ></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt"> <o:p></o:p></= span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt">The phishing bad si= tes is compiled from Phishtank data. It essentially pulls the domains from = the phishing links and puts them into a flat file for use in MailScanner.<o= :p></o:p></span></p> </div> <div> <div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif"> </span><span style=3D"font-size:11.0pt"><o:p>= </o:p></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif"> </span><span style=3D"font-size:11.0pt"><o:p>= </o:p></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif">Future plan:</span><span style=3D"font-size:11.0pt"= ><o:p></o:p></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif"> </span><span style=3D"font-size:11.0pt"><o:p>= </o:p></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif">I am looking into creating a mechanism to populate = a SQLite database with phishing links instead of just domains. I am creatin= g this for use in Mailborder, but will make the database available for MailScanner. In order to be used in MailScanner som= eone would have to write the code to use it. (I have enough to do.) Once I = create a base structure of what the database would look like, I will send o= ut an update here with a link to the SQLite database. Not sure how this would impact scanning speed at this= time. It would of course depend on how big that database gets.</span><span= style=3D"font-size:11.0pt"><o:p></o:p></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif"> </span><span style=3D"font-size:11.0pt"><o:p>= </o:p></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif">The reason why I am looking into this =85</span><sp= an style=3D"font-size:11.0pt"><o:p></o:p></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif"> </span><span style=3D"font-size:11.0pt"><o:p>= </o:p></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif">I contacted Openphish about licensing cost to get t= heir database. They want $10k per server per year for the =93light=94 versi= on. That is just nuts.</span><span style=3D"font-size:11.0pt"><o:p></o:p></= span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif"> </span><span style=3D"font-size:11.0pt"><o:p>= </o:p></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif">How you can help =85</span><span style=3D"font-size= :11.0pt"><o:p></o:p></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif"> </span><span style=3D"font-size:11.0pt"><o:p>= </o:p></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif">I need source data to compile this database. I can = get some from Phishtank and a very small amount from Openphish free sources= . I haven=92t put a whole lot of cycles into this, but if you have any suggestions, please email me.<span class=3D"apple-conv= erted-space"> </span></span><span style=3D"font-size:11.0pt"><o:p></o:= p></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif"> </span><span style=3D"font-size:11.0pt"><o:p>= </o:p></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif"> </span><span style=3D"font-size:11.0pt"><o:p>= </o:p></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif">-- </span><span style=3D"font-size:11.0pt"><o:= p></o:p></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif">Jerry Benton</span><span style=3D"font-size:11.0pt"= ><o:p></o:p></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif"><a href=3D"http://www.mailborder.com/"><span style= =3D"color:#0563C1">www.mailborder.com</span></a></span><span style=3D"font-= size:11.0pt"><o:p></o:p></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri",sans-serif">+1 843-800-8605</span><span style=3D"font-size:11.0= pt"><o:p></o:p></span></p> </div> </div> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt"> <o:p></o:p></= span></p> </div> </div> <p class=3D"MsoNormal"><span style=3D"font-size:9.0pt;font-family:Helvetica= "><br> <br> --<span class=3D"apple-converted-space"> </span><br> MailScanner mailing list<br> </span><a href=3D"mailto:mailscanner-qhrM8SXbD5JTOyd/[email protected]"><span style=3D= "font-size:9.0pt;font-family:Helvetica;color:#467886">[email protected]= lscanner.info</span></a><span style=3D"font-size:9.0pt;font-family:Helvetic= a"><br> </span><a href=3D"http://lists.mailscanner.info/mailman/listinfo/mailscanne= r"><span style=3D"font-size:9.0pt;font-family:Helvetica;color:#467886">http= ://lists.mailscanner.info/mailman/listinfo/mailscanner</span></a><o:p></o:p= ></p> </div> </blockquote> </div> <p class=3D"MsoNormal"><o:p> </o:p></p> </div> </div> </div> </div> </body> </html> --_000_DS7PR12MB8372DB0DBBA013C24C4CFFE2F4532DS7PR12MB8372namp_-- --===============5356876700151194788== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline -- MailScanner mailing list mailscanner-qhrM8SXbD5JTOyd/[email protected] http://lists.mailscanner.info/mailman/listinfo/mailscanner --===============5356876700151194788==--