BSAFE/Crypto++ CTIL Bug Report

All,

A bug has been reported (see attached message) in the BSAFE and Crypto++
(a.k.a. Free) Crypto Token Interface Libraries (CTIL) delivered in
conjunction with the v1.6 S/MIME Freeware Library (SFL).  Much thanks to
Jeff Jacoby, RSA Security, for reporting this bug.  We agree with his
analysis of the problem and his recommended fix.  We encourage all feedback
related to the SFL, CTILs and other supporting libraries.  We made the
changes in the BSAFE and Crypto++ CTIL baseline software as recommended by
Jeff in the attached message.  We successfully tested the corrected
software.  We do not plan to deliver a new release of the BSAFE and Crypto++
CTIL source code solely to fix this bug.  The bug fix will be included in
the next release of the BSAFE and Crypto++ CTIL source code.  We do not plan
to deliver patch files for this bug since the problem should be a rare
occurrence and since the solution is clearly identified in Jeff's message.
If you would like to receive the patch files, then please let me know.  

============================================
John Pawling, Director - Systems Engineering
J.G. Van Dyke & Associates, Inc;
a Wang Government Services Company
[email protected]
============================================ 

-----Original Message-----
From: Jacoby, Jeffrey <snip>
Sent: Wednesday, March 29, 2000 8:20 PM
To: [email protected]
Subject: RC2 implementation in SFL


Greetings,

I've just starting working with the SFL and I believe there 
is a problem in the RC2 message decryption implementation 
in the RSA CTIL (and possibly in the crypto++ CTIL).  I've 
discussed this with the BSAFE Crypto-C manager and he agrees
the implementaion does not correctly follow the meaning of
effective key bits as specified in RFC 2268.

The problem occurs when the effective key bit parameter 
is different from the size of the key value used to create
the B_KEY_OBJECT (used for content encryption).  RC2 allows you 
to set a B_KEY_OBJECT's KI_Item information with one number 
of key bits, e.g. 128, while specifying a different ekb value
for the A_RC2_CBC_PARAMS.effectiveBits value, e.g. 40.  

This is NOT the same as setting the KI_Item information 
with just the first 40 bits from the 128 bit key and using 
an ekb of 40.  

In the algorithm library file sm_rsa.cpp (v1.3 of the 
library, but I think v1.5 is the same in this respect), 
in the SMTI_Decrypt() method is this code (I've inserted 
comments to indicate where I believe the problem is):

   // JJ: by this point, the effective key bits value
   // JJ: has been extracted, and the BSAFE algorithm 
   // JJ: information is set

   // Create a Key Object
   if ((status = B_CreateKeyObject(&rc2Key)) != 0)
      SME_THROW(status, "B_CreateKeyObject failed", NULL);

   // Set the key data from the incoming Mek
   rc2KeyItem.len = m_rc2Params.effectiveKeyBits / 8;
//JJ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
//JJ:  This is the error.  It's possible and allowed that
//JJ:  the rc2KeyItem.len may NOT be the same as ekb/8.  It
//JJ:  very well could be greater.
   SME(rc2KeyItem.data = (unsigned char *)calloc(1,rc2KeyItem.len));
   unsigned int tmpLen=rc2KeyItem.len;
   if (tmpLen > pMEK->Length())
      tmpLen = pMEK->Length();   // ONLY get what is available.
//JJ:  We really need to get *ALL* that's available!
   memcpy(rc2KeyItem.data, pMEK->Access(), tmpLen);

   if ((status = (long)B_SetKeyInfo(rc2Key, KI_Item,
         (POINTER)&rc2KeyItem)) != 0)

The result of this is that B_DecryptFinal() may fail during
envelopedData content decryption.  The solution is to set the
B_KEY_OBJECT's KI_Item information with ALL the content encryption 
key bits/bytes extracted from the message, not just the first
m_rc2Params.effectiveKeyBits .  

While I am fairly sure I am correct on this point, it is by no 
means guaranteed :).  Please let me know if you have any comments.

Sincerely,

Jeff Jacoby, Sr. QA Engineer

RSA Security, SMDC
<snip>