openssl and compatibility with S/MIMEv3.1

Hello all,

After switching to FastCGI to fix problems with mod_perl and env
variables (thanks to Alex Vandiver's suggestion), I have found another
issue in my quest to achieve a fully functional S/MIME RT deployment.

Some of the S/MIME encrypted mails sent by Windows+Outlook users, were
not correctly decrypted by RT/openssl. The error shown is appended at
the end of the message for reference.

After some headbanging I discovered the "openssl smime" command that
RT uses is only fully compatible with S/MIMEv2 (PKCS#7) [1].
S/MIMEv3.1 (the IETF standard), standarized the use of Cryptographic
Message Syntax (CMS) format. For this, openssl developers implemented
a new command (openssl cms) that implements all S/MIME operations and
is supposed to deprecate "smime" option [2].

My tests show that "openssl cms" can decrypt all messages that I threw
at it, both PKCS#7 and CMS.
For encryption/signing, cms should produce messages which are
backwards compatible with PKCS#7, as long as you don't use some
specific options [3]. These are not used in RT, AFAIK.

I have copied rt4/lib/RT/Crypt/SMIME.pm to
rt4/local/lib/RT/Crypt/SMIME.pm, replacing "smime" with "cms" in every
command issued by RT, and it seems to work well. However, it would
require more thorough testing.

I plan to do a proper modification and pull request, but I thought it
would be better asking in the list: is there any reason I am missing
to keep using "openssl smime"?

Thank you in advance,

Marcos.

[1] https://www.openssl.org/docs/apps/smime.html#bugs
[2] http://security.stackexchange.com/questions/41399/openssl-pkcs7-vs-s-mime
[3] https://www.openssl.org/docs/apps/cms.html#compatibility_with_pkcs_7_format_


Error in RT log for S/MIMEv3.1 CMS messages:
[8282] [Mon Dec  8 15:02:04 2014] [error]: openssl exited with error
code 2 and error: Error reading S/MIME message
3493945685672:error:0D078089:asn1 encoding
routines:ASN1_ITEM_EX_D2I:missing
eoc:tasn_dec.c:470:Type=PKCS7_ENVELOPE
3493945685672:error:0D08303A:asn1 encoding
routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:751:
3493945685672:error:0D08403A:asn1 encoding
routines:ASN1_TEMPLATE_EX_D2I:nested asn1
error:tasn_dec.c:579:Field=d.enveloped, Type=PKCS7
3493945685672:error:0D0D106E:asn1 encoding
routines:B64_READ_ASN1:decode error:asn_mime.c:193:
3493945685672:error:0D0D40CB:asn1 encoding
routines:SMIME_read_ASN1:asn1 parse error:asn_mime.c:528:
(/opt/rt4/sbin/../lib/RT/Crypt/SMIME.pm:604)