Re: JDK11 & BC & TLS with RSASSA-PSS signature

There seem to be a few issues around setParameter() in Java 11.

The convention on this has always been to call setParameter() first.

We're going to treat this one as a bug, but I would suggest filing this
as either a "bug" or a "request for enhancement". I can understand how
the current maintainers of the JSSE would not be aware of it, but the
convention was established 20 years ago, and the comment above the
modified code in SignatureScheme.java makes absolutely no sense. It
looks like someone made the change will trying to fix something else and
got confused as to which change actually solved the problem.

Regards,

David

On 21/12/18 2:39 am, MSKnete-S0/[email protected] wrote:
> Hello all,
>  
> I run into an issue, when using JDK11, BC and TLS with RSASSA-PSS
> signature scheme. From JDK10 to JDK11 the handshake classes have been
> changed and especially the JDK class
> 'src/java.base/share/classes/sun/security/ssl/SignatureScheme.java'
> calls the signer object in an unfavourable manner.
>  
> Instead of calling setParameter() first and then initSign(), JDK11
> does it the other way around (SignatureScheme.java, line 473/480).
>  
> Obviously, BC expect calling these two methods in the opposite order
> (org.bouncycastle.jcajce.provider.asymmetric.rsa.PSSSignatureSpi.java).
>  
> My remaining question: Who does it right? BC or JDK? Which side must
> be fixed?
>  
> I also attached a small sample which demonstrates the issue by
> comparing the BC implementation with the SunRsaSign implementation.
>  
> Further I found another mentioning of this behavior on the openjdk
> mailing list at
> http://mail.openjdk.java.net/pipermail/security-dev/2018-September/018265.html
> but sadly without any continuative response.
>  
> Any suggestion are welcome, to get RSASSA-PSS and BC working with JDK11.
>  
> Kind regards,
> Michael
>
> $ java -version
> openjdk version "11.0.1" 2018-10-16
> OpenJDK Runtime Environment 18.9 (build 11.0.1+13)
> OpenJDK 64-Bit Server VM 18.9 (build 11.0.1+13, mixed mode)
>  
> $ javac -cp bcprov-jdk15on-160.jar Demo.java
> $ java -cp bcprov-jdk15on-160.jar Demo
> BCprov: [...]
> SunJCE: [...]
> FAIL: Signatures are not equal!
>  
> ---------------------
> Sample Source (Demo.java):
> ---------------------
> import java.security.KeyPair;
> import java.security.KeyPairGenerator;
> import java.security.PrivateKey;
> import java.security.SecureRandom;
> import java.security.Security;
> import java.security.Signature;
> import java.security.spec.MGF1ParameterSpec;
> import java.security.spec.PSSParameterSpec;
> import java.util.Arrays;
>
> public class Demo {
>  
>   public static void main(final String[] args) throws Exception {
>     Security.insertProviderAt(new
> org.bouncycastle.jce.provider.BouncyCastleProvider(), 2);
>  
>     KeyPairGenerator keyPairGenerator =
> KeyPairGenerator.getInstance("RSA");
>     keyPairGenerator.initialize(2048);
>     KeyPair keyPair = keyPairGenerator.generateKeyPair();
>     PrivateKey privateKey = keyPair.getPrivate();
>  
>     SecureRandom prngBC = SecureRandom.getInstance("SHA1PRNG");
>     prngBC.setSeed(12345678L); //we need equal randomness, dont do
> this in production!
>     Signature signerBC = Signature.getInstance("RSASSA-PSS", "BC");
>  
>     // this breaks BC, since setParameter() is invoked AFTER init()
>     signerBC.initSign(privateKey, prngBC);
>     signerBC.setParameter(new PSSParameterSpec("SHA-512", "MGF1", new
> MGF1ParameterSpec("SHA-512"), 64, 1));
>     byte[] resultBC = signerBC.sign();
>  
>     SecureRandom prngSun = SecureRandom.getInstance("SHA1PRNG");
>     prngSun.setSeed(12345678L); //we need equal randomness, dont do
> this in production!
>     Signature signerSun = Signature.getInstance("RSASSA-PSS",
> "SunRsaSign");
>     signerSun.initSign(privateKey, prngSun);
>     signerSun.setParameter(new PSSParameterSpec("SHA-512", "MGF1", new
> MGF1ParameterSpec("SHA-512"), 64, 1));
>     byte[] resultSun = signerSun.sign();
>  
>     System.out.println("BCprov: " + Arrays.toString(resultBC));
>     System.out.println("SunJCE: " + Arrays.toString(resultSun));
>     if (Arrays.equals(resultBC, resultSun)) {
>       System.out.println("OK: Signatures are equal!");
>     } else {
>       System.err.println("FAIL: Signatures are not equal!");
>     }
>   }
> }