[M-git] Mahogany sources repository. branch master updated. v0.67-783-g1b285d6f
Vadim Zeitlin via Mahogany-cvsupdates <[email protected]> Sun, 21 Jul 2019 12:36:54 +0000
| Newsgroups | gmane.mail.mahogany.cvs |
|---|---|
| Message-ID | <[email protected]> |
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "Mahogany sources repository.".
The branch, master has been updated
via 1b285d6ff380a49571eb766a03257670551719d4 (commit)
via 7089094b95b7858f263b986f49bdf8409aeae468 (commit)
via f70237fdb3ea7b2423c6ac59443f8d488639c0e1 (commit)
via a6c1e3e87239a9bc9a265db11ff523abd43907a4 (commit)
via 9b5b3a508259c5f05f2fec6a650f552a57177c01 (commit)
via 0c8afa393de9d49f0005bf4225c79a230582579e (commit)
via 61864a6f657f02ceee767bac57d55d008f3bd4a6 (commit)
from 7a2637cc0293354853e42b6098864305be166d60 (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
- Log -----------------------------------------------------------------
commit 1b285d6ff380a49571eb766a03257670551719d4
Author: Vadim Zeitlin <[email protected]>
Date: Tue Jul 16 15:03:38 2019 +0200
Prefer using lax UTF-8 decoding to Latin-1 for UTF-8 MIME parts
In 2019 it's vastly more likely that a message part using UTF-8 encoding
is actually encoded in UTF-8 and just has some non-UTF-8 bit appended to
it at the end, than it doesn't use UTF-8 at all (as happened 20 years
with some completely broken email-sending scripts that simple added
charset="UTF-8" to Content-Type without worrying about the actual
encoding of the data).
So stop replacing UTF-8 with Latin-1 at MimePart level and try to decode
the contents as UTF-8, but just ignore the non-UTF-8 characters instead.
From the user point of view, it means that the message with a non-UTF-8
trailer will now appear almost correctly by default, instead of having
to manually select "Language|UTF-8" to be able to see it correctly.
diff --git a/src/mail/MimePartCCBase.cpp b/src/mail/MimePartCCBase.cpp
index 72d7ae7a..5aff0c7c 100644
--- a/src/mail/MimePartCCBase.cpp
+++ b/src/mail/MimePartCCBase.cpp
@@ -284,28 +284,6 @@ wxFontEncoding MimePartCCBase::GetTextEncoding() const
{
m_encoding = wxFONTENCODING_ISO8859_1;
}
-
- // special case: many broken programs generate messages with UTF-8
- // charset but without properly encoding the contents in UTF-8 which
- // results in "s" being empty and not showing the text parts at all
- //
- // so if the conversion failed, try to show the text at least somehow
- // using latin1 (and the user will be able to change the encoding
- // manually from the menu later which would be impossible if we returned
- // an empty string from GetTextContent())
- if ( m_encoding == wxFONTENCODING_UTF8 ||
- m_encoding == wxFONTENCODING_UTF7 )
- {
- // check if we really have valid UTF-x
- unsigned long len;
- const char *p = reinterpret_cast<const char *>(GetContent(&len));
-
- if ( p &&
- wxCSConv(m_encoding).ToWChar(NULL, 0, p, len) == wxCONV_FAILED )
- {
- m_encoding = wxFONTENCODING_ISO8859_1;
- }
- }
}
return m_encoding;
@@ -496,7 +474,7 @@ String MimePartCCBase::GetTextContent() const
wxString s;
if ( p )
{
- s = wxString(p, wxCSConv(GetTextEncoding()), len);
+ s = MIME::DecodeText(p, len, GetTextEncoding());
}
return s;
commit 7089094b95b7858f263b986f49bdf8409aeae468
Author: Vadim Zeitlin <[email protected]>
Date: Tue Jul 16 15:03:03 2019 +0200
Refactor UTF-8 non-strict decoding to a separate function
Add MIME::DecodeText() for "lossless" decoding of UTF-8.
No real changes.
diff --git a/include/mail/MimeDecode.h b/include/mail/MimeDecode.h
index b62b1c90..c20a3f0f 100644
--- a/include/mail/MimeDecode.h
+++ b/include/mail/MimeDecode.h
@@ -81,6 +81,15 @@ EncodeHeader(const wxString& in, wxFontEncoding enc = wxFONTENCODING_SYSTEM);
*/
String DecodeHeader(const String& in, wxFontEncoding *encoding = NULL);
+/**
+ Helper for decoding the given data using the specified encoding.
+
+ This method tries hard to return something useful and ignores invalid
+ characters when decoding using UTF-8 instead of failing and returning an
+ empty string, which is not useful.
+ */
+String DecodeText(const char *p, size_t len, wxFontEncoding enc);
+
} // namespace MIME
#endif // M_MAIL_MIMEDECODE_H
diff --git a/src/classes/MessageView.cpp b/src/classes/MessageView.cpp
index 23b6fcc4..a73f5d37 100644
--- a/src/classes/MessageView.cpp
+++ b/src/classes/MessageView.cpp
@@ -176,19 +176,7 @@ RecodeText(String *text, wxFontEncoding encSrc, wxFontEncoding encDst)
const wxCharBuffer textMB(text->mb_str(wxCSConv(encSrc)));
CHECK_RET( textMB, "string not in source encoding?" );
- // Special case of using UTF-8: it often happens that a message encoded in
- // UTF-8 has some trailer with Latin-1 appended to it. In this case, try to
- // recover as much of UTF-8 text as possible.
- if ( encDst == wxFONTENCODING_UTF8 )
- {
- *text = String(textMB, wxMBConvUTF8(wxMBConvUTF8::MAP_INVALID_UTF8_TO_PUA));
- }
- else
- {
- String textNew = String(textMB, wxCSConv(encDst));
- if ( !textNew.empty() )
- text->swap(textNew);
- }
+ *text = MIME::DecodeText(textMB, textMB.length(), encDst);
}
#else // !wxUSE_UNICODE
diff --git a/src/mail/MimeDecode.cpp b/src/mail/MimeDecode.cpp
index c1d9c95c..f8f511f1 100644
--- a/src/mail/MimeDecode.cpp
+++ b/src/mail/MimeDecode.cpp
@@ -738,3 +738,18 @@ wxCharBuffer MIME::EncodeHeader(const String& in, wxFontEncoding enc)
return headerEnc.ToAscii();
}
+
+String MIME::DecodeText(const char *p, size_t len, wxFontEncoding enc)
+{
+ // Special case of using UTF-8: it often happens that a message encoded in
+ // UTF-8 has some trailer with Latin-1 appended to it. In this case, try to
+ // recover as much of UTF-8 text as possible.
+ if ( enc == wxFONTENCODING_UTF8 )
+ {
+ return String(p, wxMBConvUTF8(wxMBConvUTF8::MAP_INVALID_UTF8_TO_PUA), len);
+ }
+ else
+ {
+ return String(p, wxCSConv(enc), len);
+ }
+}
commit f70237fdb3ea7b2423c6ac59443f8d488639c0e1
Author: Vadim Zeitlin <[email protected]>
Date: Tue Jul 16 14:41:08 2019 +0200
Allow viewing not quite UTF-8 messages using UTF-8
Ignore invalid characters when decoding the message text using UTF-8,
this can be useful when the message is mostly UTF-8, but just happens to
contain some trailing junk not using UTF-8 at the end.
And it's definitely better to show at least something when the user
selects "UTF-8" from the "Language" menu rather than nothing at all, as
happened before.
diff --git a/src/classes/MessageView.cpp b/src/classes/MessageView.cpp
index aa936579..23b6fcc4 100644
--- a/src/classes/MessageView.cpp
+++ b/src/classes/MessageView.cpp
@@ -176,7 +176,19 @@ RecodeText(String *text, wxFontEncoding encSrc, wxFontEncoding encDst)
const wxCharBuffer textMB(text->mb_str(wxCSConv(encSrc)));
CHECK_RET( textMB, "string not in source encoding?" );
- *text = String(textMB, wxCSConv(encDst));
+ // Special case of using UTF-8: it often happens that a message encoded in
+ // UTF-8 has some trailer with Latin-1 appended to it. In this case, try to
+ // recover as much of UTF-8 text as possible.
+ if ( encDst == wxFONTENCODING_UTF8 )
+ {
+ *text = String(textMB, wxMBConvUTF8(wxMBConvUTF8::MAP_INVALID_UTF8_TO_PUA));
+ }
+ else
+ {
+ String textNew = String(textMB, wxCSConv(encDst));
+ if ( !textNew.empty() )
+ text->swap(textNew);
+ }
}
#else // !wxUSE_UNICODE
commit a6c1e3e87239a9bc9a265db11ff523abd43907a4
Author: Vadim Zeitlin <[email protected]>
Date: Tue Jul 16 14:02:31 2019 +0200
Use UTF-8 as fallback encoding for MIME headers directly
This makes more sense than using the local system encoding, which is not
really relevant for the headers being sent out.
diff --git a/src/mail/MimeDecode.cpp b/src/mail/MimeDecode.cpp
index bce5ec4a..c1d9c95c 100644
--- a/src/mail/MimeDecode.cpp
+++ b/src/mail/MimeDecode.cpp
@@ -643,16 +643,12 @@ wxCharBuffer MIME::EncodeHeader(const String& in, wxFontEncoding enc)
if ( !NeedsEncoding(in) )
return in.ToAscii();
- // decide about the encoding to use if none specified
- if ( enc == wxFONTENCODING_SYSTEM )
+ // If we were given an explicit encoding to use, check if we can use it, and
+ // if not fall back to the same UTF-8 (which can always be used) as we use
+ // by default if no encoding was specified in the first place.
+ if ( enc == wxFONTENCODING_SYSTEM ||
+ wxCSConv(enc).FromWChar(NULL, 0, in.wc_str(wxConvLibc)) == wxCONV_FAILED )
{
- // try to use the user current encoding first
- enc = wxLocale::GetSystemEncoding();
- }
-
- if ( wxCSConv(enc).FromWChar(NULL, 0, in.wc_str(wxConvLibc)) == wxCONV_FAILED )
- {
- // but if we can't encode with it, fall back to UTF-8 as it never fails
enc = wxFONTENCODING_UTF8;
}
commit 9b5b3a508259c5f05f2fec6a650f552a57177c01
Author: Vadim Zeitlin <[email protected]>
Date: Tue Jul 16 14:00:28 2019 +0200
Remove unnecessary inclusion of a private wx header
No real changes.
diff --git a/src/modules/TextViewer.cpp b/src/modules/TextViewer.cpp
index bd7bac69..9e7d0291 100644
--- a/src/modules/TextViewer.cpp
+++ b/src/modules/TextViewer.cpp
@@ -37,10 +37,6 @@
#include <wx/html/htmprint.h> // for wxHtmlEasyPrinting
-#ifdef __WXMSW__
- #include <wx/msw/private.h>
-#endif // __WXMSW__
-
// only Win32 supports URLs in the text control natively so far, define this to
// use this possibility
//
commit 0c8afa393de9d49f0005bf4225c79a230582579e
Author: Vadim Zeitlin <[email protected]>
Date: Thu Mar 14 22:14:29 2019 +0100
Avoid assertion failures when reloading SSL DLLs
If InitSSL() failed due to a missing function in either of SSL shared
libraries, then calling it again would result in an assertion failure
(in debug build only, of course) from wxDynamicLibrary::Load().
Fix this by explicitly unloading the libraries in case of failure.
diff --git a/src/util/ssl.cpp b/src/util/ssl.cpp
index 417447cb..4483f4ef 100644
--- a/src/util/ssl.cpp
+++ b/src/util/ssl.cpp
@@ -394,6 +394,12 @@ bool InitSSL(void) /* FIXME: MT */
return true;
error:
+ // Unload the DLLs to avoid assertion failures when calling Load() on them
+ // again if we reattempt SSL initialization later (note that Unload() itself
+ // doesn't assert and just doesn't do anything if the DLL is not loaded).
+ gs_dllSll.Unload();
+ gs_dllCrypto.Unload();
+
if ( !s_errMsgGiven )
{
ERRORMESSAGE((_("SSL authentication is not available.")));
commit 61864a6f657f02ceee767bac57d55d008f3bd4a6
Author: Vadim Zeitlin <[email protected]>
Date: Thu Mar 14 22:10:03 2019 +0100
Remove SSL_state not existing any more in OpenSSL 1.1
It seems to have never been used, so there is no need to replace it with
SSL_get_state().
diff --git a/src/util/ssl.cpp b/src/util/ssl.cpp
index 8d56305d..417447cb 100644
--- a/src/util/ssl.cpp
+++ b/src/util/ssl.cpp
@@ -357,7 +357,6 @@ bool InitSSL(void) /* FIXME: MT */
SSL_LOOKUP(SSL_CTX_set_default_verify_paths);
SSL_LOOKUP(SSL_set_bio);
SSL_LOOKUP(SSL_set_connect_state);
- SSL_LOOKUP(SSL_state);
SSL_LOOKUP(SSL_ctrl);
SSL_LOOKUP(TLSv1_server_method);
SSL_LOOKUP(SSL_CTX_set_cipher_list);
-----------------------------------------------------------------------
Summary of changes:
include/mail/MimeDecode.h | 9 +++++++++
src/classes/MessageView.cpp | 2 +-
src/mail/MimeDecode.cpp | 29 ++++++++++++++++++++---------
src/mail/MimePartCCBase.cpp | 24 +-----------------------
src/modules/TextViewer.cpp | 4 ----
src/util/ssl.cpp | 7 ++++++-
6 files changed, 37 insertions(+), 38 deletions(-)
hooks/post-receive
--
Mahogany sources repository.