Re: Phishing Script and Server Update & Future Plans
Bogdan-Stefan Rotariu <[email protected]> Wed, 6 Nov 2024 16:26:32 +0200
| Newsgroups | gmane.mail.virus.mailscanner |
|---|---|
| Message-ID | <[email protected]> |
--===============1538451547840931104==
Content-Type: multipart/alternative;
boundary="Apple-Mail=_4EC8B8DF-BDF8-4041-B1A8-1849AF1BB7F7"
--Apple-Mail=_4EC8B8DF-BDF8-4041-B1A8-1849AF1BB7F7
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
charset=utf-8
Thank you for the recent update. I would like to suggest two changes:
1. Since we have already set the PATH, we can simply use ms-peek without =
specifying the full path (/usr/sbin/ms-peek).
2. Regarding the enforcement of HTTPS, we have encountered issues with =
bad .gz files. Currently, the only fix is to remove them manually. I =
propose that the script either overwrites these .gz files or removes =
them before downloading new ones to prevent this issue.
gzip: /usr/mailscanner/etc/phishing.safe.sites.conf.master.gz: not in =
gzip format
I have added a workaround to our script:
if [ -d $CONFIGDIR ]; then
cd $CONFIGDIR
#delete .gz old gz files if left behind
find "$CONFIGDIR" -name "*.gz" -type f -exec rm -f {} +
Thanks,
--
Bogdan-Stefan Rotariu
> On 6 Nov 2024, at 14:20, Jerry Benton <jerry.benton-OT/[email protected]> =
wrote:
>=20
> I have updated the cron script for phishing bad/safe sites to v0.3.1. =
It is available here:https://phishing.mailscanner.info =
<https://phishing.mailscanner.info/>
> =20
> The phishing update server has been updated to use https, but will not =
force that connection and will now accept http.
> =20
> Several previous automatic bad site scrubs were removed for the google =
domain and several subdomains. This means you will need to add them to =
the phishing safe sites custom file to retain links to google domains.
> =20
> Uncompressed versions of the bad/safe sites files are no longer =
available. If you have an old script trying to download those files, it =
will fail. You will need to use the newer update script or modify any =
custom scripts to download the compressed versions and then extract them =
for use.=20
> =20
> The phishing bad sites is compiled from Phishtank data. It essentially =
pulls the domains from the phishing links and puts them into a flat file =
for use in MailScanner.
> =20
> =20
> Future plan:
> =20
> I am looking into creating a mechanism to populate a SQLite database =
with phishing links instead of just domains. I am creating this for use =
in Mailborder, but will make the database available for MailScanner. In =
order to be used in MailScanner someone would have to write the code to =
use it. (I have enough to do.) Once I create a base structure of what =
the database would look like, I will send out an update here with a link =
to the SQLite database. Not sure how this would impact scanning speed at =
this time. It would of course depend on how big that database gets.
> =20
> The reason why I am looking into this =E2=80=A6
> =20
> I contacted Openphish about licensing cost to get their database. They =
want $10k per server per year for the =E2=80=9Clight=E2=80=9D version. =
That is just nuts.
> =20
> How you can help =E2=80=A6
> =20
> I need source data to compile this database. I can get some from =
Phishtank and a very small amount from Openphish free sources. I =
haven=E2=80=99t put a whole lot of cycles into this, but if you have any =
suggestions, please email me.=20
> =20
> =20
> --=20
> Jerry Benton
> www.mailborder.com <http://www.mailborder.com/>
> +1 843-800-8605
> =20
>=20
>=20
> --=20
> MailScanner mailing list
> mailscanner-qhrM8SXbD5JTOyd/[email protected] =
<mailto:mailscanner-qhrM8SXbD5JTOyd/[email protected]>
> http://lists.mailscanner.info/mailman/listinfo/mailscanner
--Apple-Mail=_4EC8B8DF-BDF8-4041-B1A8-1849AF1BB7F7
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
charset=utf-8
<html><head><meta http-equiv=3D"content-type" content=3D"text/html; =
charset=3Dutf-8"></head><body style=3D"overflow-wrap: break-word; =
-webkit-nbsp-mode: space; line-break: after-white-space;"><div =
style=3D"caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);">Thank you for =
the recent update. I would like to suggest two changes:</div><div =
style=3D"caret-color: rgb(0, 0, 0); color: rgb(0, 0, =
0);"><br>1. Since we have already set the PATH, we can simply use =
ms-peek without specifying the full path (/usr/sbin/ms-peek).</div><div =
style=3D"caret-color: rgb(0, 0, 0); color: rgb(0, 0, =
0);">2. Regarding the enforcement of HTTPS, we have encountered =
issues with bad .gz files. Currently, the only fix is to remove them =
manually. I propose that the script either overwrites these .gz files or =
removes them before downloading new ones to prevent this =
issue.</div><span style=3D"caret-color: rgb(0, 0, 0); color: rgb(0, 0, =
0);"><br></span><span style=3D"caret-color: rgb(0, 0, 0); color: rgb(0, =
0, 0);">gzip: /usr/mailscanner/etc/phishing.safe.sites.conf.master.gz: =
not in gzip format</span><div style=3D"caret-color: rgb(0, 0, 0); color: =
rgb(0, 0, 0);"><br>I have added a workaround to our script:</div><div =
style=3D"caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);"><br></div><div =
style=3D"caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);">if [ -d =
$CONFIGDIR ]; then<br> cd $CONFIGDIR<br>#delete .gz old gz =
files if left behind<br>find "$CONFIGDIR" -name "*.gz" -type f -exec rm =
-f {} +<br><br>Thanks,</div><div style=3D"caret-color: rgb(0, 0, 0); =
color: rgb(0, 0, 0);"><br></div><div>
<div style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space;"><div style=3D"color: rgb(0, 0, =
0); font-family: Helvetica; font-size: 12px; font-style: normal; =
font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
orphans: auto; text-align: start; text-indent: 0px; text-transform: =
none; white-space: normal; widows: auto; word-spacing: 0px; =
-webkit-text-stroke-width: 0px;">--</div><div style=3D"color: rgb(0, 0, =
0); font-family: Helvetica; font-size: 12px; font-style: normal; =
font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
orphans: auto; text-align: start; text-indent: 0px; text-transform: =
none; white-space: normal; widows: auto; word-spacing: 0px; =
-webkit-text-stroke-width: 0px;">Bogdan-Stefan Rotariu</div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline">
</div>
<div><br><blockquote type=3D"cite"><div>On 6 Nov 2024, at 14:20, Jerry =
Benton <jerry.benton-OT/[email protected]> wrote:</div><br =
class=3D"Apple-interchange-newline"><div><meta charset=3D"UTF-8"><div =
class=3D"WordSection1" style=3D"page: WordSection1; caret-color: rgb(0, =
0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; =
font-variant-caps: normal; font-weight: 400; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;"><div style=3D"margin: 0in; font-size: 11pt; =
font-family: Aptos, sans-serif;">I have updated the cron script for =
phishing bad/safe sites to v0.3.1. It is available here:<a =
href=3D"https://phishing.mailscanner.info/" style=3D"color: rgb(70, 120, =
134); text-decoration: =
underline;">https://phishing.mailscanner.info</a><o:p></o:p></div><div =
style=3D"margin: 0in; font-size: 11pt; font-family: Aptos, =
sans-serif;"><o:p> </o:p></div><div style=3D"margin: 0in; =
font-size: 11pt; font-family: Aptos, sans-serif;">The phishing update =
server has been updated to use https, but will not force that connection =
and will now accept http.<o:p></o:p></div><div style=3D"margin: 0in; =
font-size: 11pt; font-family: Aptos, =
sans-serif;"><o:p> </o:p></div><div style=3D"margin: 0in; =
font-size: 11pt; font-family: Aptos, sans-serif;">Several previous =
automatic bad site scrubs were removed for the google domain and several =
subdomains. This means you will need to add them to the phishing safe =
sites custom file to retain links to google =
domains.<o:p></o:p></div><div style=3D"margin: 0in; font-size: 11pt; =
font-family: Aptos, sans-serif;"> <o:p></o:p></div><div =
style=3D"margin: 0in; font-size: 11pt; font-family: Aptos, =
sans-serif;">Uncompressed versions of the bad/safe sites files are no =
longer available. If you have an old script trying to download those =
files, it will fail. You will need to use the newer update script or =
modify any custom scripts to download the compressed versions and then =
extract them for use.<span =
class=3D"Apple-converted-space"> </span><o:p></o:p></div><div =
style=3D"margin: 0in; font-size: 11pt; font-family: Aptos, =
sans-serif;"><o:p> </o:p></div><div style=3D"margin: 0in; =
font-size: 11pt; font-family: Aptos, sans-serif;">The phishing bad sites =
is compiled from Phishtank data. It essentially pulls the domains from =
the phishing links and puts them into a flat file for use in =
MailScanner.<o:p></o:p></div><div><div><div style=3D"margin: 0in; =
font-size: 11pt; font-family: Aptos, sans-serif;"><span =
style=3D"font-family: Calibri, =
sans-serif;"><o:p> </o:p></span></div><div style=3D"margin: 0in; =
font-size: 11pt; font-family: Aptos, sans-serif;"><span =
style=3D"font-family: Calibri, =
sans-serif;"><o:p> </o:p></span></div><div style=3D"margin: 0in; =
font-size: 11pt; font-family: Aptos, sans-serif;"><span =
style=3D"font-family: Calibri, sans-serif;">Future =
plan:<o:p></o:p></span></div><div style=3D"margin: 0in; font-size: 11pt; =
font-family: Aptos, sans-serif;"><span style=3D"font-family: Calibri, =
sans-serif;"><o:p> </o:p></span></div><div style=3D"margin: 0in; =
font-size: 11pt; font-family: Aptos, sans-serif;"><span =
style=3D"font-family: Calibri, sans-serif;">I am looking into creating a =
mechanism to populate a SQLite database with phishing links instead of =
just domains. I am creating this for use in Mailborder, but will make =
the database available for MailScanner. In order to be used in =
MailScanner someone would have to write the code to use it. (I have =
enough to do.) Once I create a base structure of what the database would =
look like, I will send out an update here with a link to the SQLite =
database. Not sure how this would impact scanning speed at this time. It =
would of course depend on how big that database =
gets.<o:p></o:p></span></div><div style=3D"margin: 0in; font-size: 11pt; =
font-family: Aptos, sans-serif;"><span style=3D"font-family: Calibri, =
sans-serif;"><o:p> </o:p></span></div><div style=3D"margin: 0in; =
font-size: 11pt; font-family: Aptos, sans-serif;"><span =
style=3D"font-family: Calibri, sans-serif;">The reason why I am looking =
into this =E2=80=A6<o:p></o:p></span></div><div style=3D"margin: 0in; =
font-size: 11pt; font-family: Aptos, sans-serif;"><span =
style=3D"font-family: Calibri, =
sans-serif;"><o:p> </o:p></span></div><div style=3D"margin: 0in; =
font-size: 11pt; font-family: Aptos, sans-serif;"><span =
style=3D"font-family: Calibri, sans-serif;">I contacted Openphish about =
licensing cost to get their database. They want $10k per server per year =
for the =E2=80=9Clight=E2=80=9D version. That is just =
nuts.<o:p></o:p></span></div><div style=3D"margin: 0in; font-size: 11pt; =
font-family: Aptos, sans-serif;"><span style=3D"font-family: Calibri, =
sans-serif;"><o:p> </o:p></span></div><div style=3D"margin: 0in; =
font-size: 11pt; font-family: Aptos, sans-serif;"><span =
style=3D"font-family: Calibri, sans-serif;">How you can help =
=E2=80=A6<o:p></o:p></span></div><div style=3D"margin: 0in; font-size: =
11pt; font-family: Aptos, sans-serif;"><span style=3D"font-family: =
Calibri, sans-serif;"><o:p> </o:p></span></div><div style=3D"margin: =
0in; font-size: 11pt; font-family: Aptos, sans-serif;"><span =
style=3D"font-family: Calibri, sans-serif;">I need source data to =
compile this database. I can get some from Phishtank and a very small =
amount from Openphish free sources. I haven=E2=80=99t put a whole lot of =
cycles into this, but if you have any suggestions, please email me.<span =
class=3D"Apple-converted-space"> </span><o:p></o:p></span></div><div =
style=3D"margin: 0in; font-size: 11pt; font-family: Aptos, =
sans-serif;"><span style=3D"font-family: Calibri, =
sans-serif;"><o:p> </o:p></span></div><div style=3D"margin: 0in; =
font-size: 11pt; font-family: Aptos, sans-serif;"><span =
style=3D"font-family: Calibri, =
sans-serif;"><o:p> </o:p></span></div><div style=3D"margin: 0in; =
font-size: 11pt; font-family: Aptos, sans-serif;"><span =
style=3D"font-family: Calibri, =
sans-serif;">-- <o:p></o:p></span></div><div style=3D"margin: 0in; =
font-size: 11pt; font-family: Aptos, sans-serif;"><span =
style=3D"font-family: Calibri, sans-serif;">Jerry =
Benton<o:p></o:p></span></div><div style=3D"margin: 0in; font-size: =
11pt; font-family: Aptos, sans-serif;"><span style=3D"font-family: =
Calibri, sans-serif;"><a href=3D"http://www.mailborder.com/" =
style=3D"color: rgb(70, 120, 134); text-decoration: underline;"><span =
style=3D"color: rgb(5, 99, =
193);">www.mailborder.com</span></a><o:p></o:p></span></div><div =
style=3D"margin: 0in; font-size: 11pt; font-family: Aptos, =
sans-serif;"><span style=3D"font-family: Calibri, sans-serif;">+1 =
843-800-8605<o:p></o:p></span></div></div></div><div style=3D"margin: =
0in; font-size: 11pt; font-family: Aptos, =
sans-serif;"><o:p> </o:p></div></div><br style=3D"caret-color: =
rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: =
normal; font-variant-caps: normal; font-weight: 400; letter-spacing: =
normal; text-align: start; text-indent: 0px; text-transform: none; =
white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;"><br style=3D"caret-color: rgb(0, 0, 0); =
font-family: Helvetica; font-size: 12px; font-style: normal; =
font-variant-caps: normal; font-weight: 400; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;"><span style=3D"caret-color: rgb(0, 0, 0); =
font-family: Helvetica; font-size: 12px; font-style: normal; =
font-variant-caps: normal; font-weight: 400; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none; float: none; display: inline !important;">--<span =
class=3D"Apple-converted-space"> </span></span><br =
style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: =
12px; font-style: normal; font-variant-caps: normal; font-weight: 400; =
letter-spacing: normal; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; text-decoration: none;"><span =
style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: =
12px; font-style: normal; font-variant-caps: normal; font-weight: 400; =
letter-spacing: normal; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; text-decoration: none; float: none; =
display: inline !important;">MailScanner mailing list</span><br =
style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: =
12px; font-style: normal; font-variant-caps: normal; font-weight: 400; =
letter-spacing: normal; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; text-decoration: none;"><a =
href=3D"mailto:mailscanner-qhrM8SXbD5JTOyd/[email protected]" style=3D"color: =
rgb(70, 120, 134); text-decoration: underline; font-family: Helvetica; =
font-size: 12px; font-style: normal; font-variant-caps: normal; =
font-weight: 400; letter-spacing: normal; orphans: auto; text-align: =
start; text-indent: 0px; text-transform: none; white-space: normal; =
widows: auto; word-spacing: 0px; -webkit-text-stroke-width: =
0px;">mailscanner-qhrM8SXbD5JTOyd/[email protected]</a><br style=3D"caret-color: =
rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: =
normal; font-variant-caps: normal; font-weight: 400; letter-spacing: =
normal; text-align: start; text-indent: 0px; text-transform: none; =
white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;"><a =
href=3D"http://lists.mailscanner.info/mailman/listinfo/mailscanner" =
style=3D"color: rgb(70, 120, 134); text-decoration: underline; =
font-family: Helvetica; font-size: 12px; font-style: normal; =
font-variant-caps: normal; font-weight: 400; letter-spacing: normal; =
orphans: auto; text-align: start; text-indent: 0px; text-transform: =
none; white-space: normal; widows: auto; word-spacing: 0px; =
-webkit-text-stroke-width: =
0px;">http://lists.mailscanner.info/mailman/listinfo/mailscanner</a></div>=
</blockquote></div><br></body></html>=
--Apple-Mail=_4EC8B8DF-BDF8-4041-B1A8-1849AF1BB7F7--
--===============1538451547840931104==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
--
MailScanner mailing list
mailscanner-qhrM8SXbD5JTOyd/[email protected]
http://lists.mailscanner.info/mailman/listinfo/mailscanner
--===============1538451547840931104==--