JDK11 & BC & TLS with RSASSA-PSS signature

Hello all,

I run into an issue, when using JDK11, BC and TLS with RSASSA-PSS signature scheme. From JDK10 to JDK11 the handshake classes have been changed and especially the JDK class 'src/java.base/share/classes/sun/security/ssl/SignatureScheme.java' calls the signer object in an unfavourable manner.

Instead of calling setParameter() first and then initSign(), JDK11 does it the other way around (SignatureScheme.java, line 473/480).

Obviously, BC expect calling these two methods in the opposite order (org.bouncycastle.jcajce.provider.asymmetric.rsa.PSSSignatureSpi.java).

My remaining question: Who does it right? BC or JDK? Which side must be fixed?

I also attached a small sample which demonstrates the issue by comparing the BC implementation with the SunRsaSign implementation.

Further I found another mentioning of this behavior on the openjdk mailing list at http://mail.openjdk.java.net/pipermail/security-dev/2018-September/018265.html but sadly without any continuative response.

Any suggestion are welcome, to get RSASSA-PSS and BC working with JDK11.

Kind regards,

Michael

$ java -version

openjdk version "11.0.1" 2018-10-16

OpenJDK Runtime Environment 18.9 (build 11.0.1+13)

OpenJDK 64-Bit Server VM 18.9 (build 11.0.1+13, mixed mode)

$ javac -cp bcprov-jdk15on-160.jar Demo.java

$ java -cp bcprov-jdk15on-160.jar Demo

BCprov: [...]

SunJCE: [...]

FAIL: Signatures are not equal!

---------------------

Sample Source (Demo.java):

---------------------

import java.security.KeyPair;

import java.security.KeyPairGenerator;

import java.security.PrivateKey;

import java.security.SecureRandom;

import java.security.Security;

import java.security.Signature;

import java.security.spec.MGF1ParameterSpec;

import java.security.spec.PSSParameterSpec;

import java.util.Arrays;

public class Demo {

public static void main(final String[] args) throws Exception {

Security.insertProviderAt(new org.bouncycastle.jce.provider.BouncyCastleProvider(), 2);

KeyPairGenerator keyPairGenerator = KeyPairGenerator.getInstance("RSA");

keyPairGenerator.initialize(2048);

KeyPair keyPair = keyPairGenerator.generateKeyPair();

PrivateKey privateKey = keyPair.getPrivate();

SecureRandom prngBC = SecureRandom.getInstance("SHA1PRNG");

prngBC.setSeed(12345678L); //we need equal randomness, dont do this in production!

Signature signerBC = Signature.getInstance("RSASSA-PSS", "BC");

// this breaks BC, since setParameter() is invoked AFTER init()

signerBC.initSign(privateKey, prngBC);

signerBC.setParameter(new PSSParameterSpec("SHA-512", "MGF1", new MGF1ParameterSpec("SHA-512"), 64, 1));

byte[] resultBC = signerBC.sign();

SecureRandom prngSun = SecureRandom.getInstance("SHA1PRNG");

prngSun.setSeed(12345678L); //we need equal randomness, dont do this in production!

Signature signerSun = Signature.getInstance("RSASSA-PSS", "SunRsaSign");

signerSun.initSign(privateKey, prngSun);

signerSun.setParameter(new PSSParameterSpec("SHA-512", "MGF1", new MGF1ParameterSpec("SHA-512"), 64, 1));

byte[] resultSun = signerSun.sign();

System.out.println("BCprov: " + Arrays.toString(resultBC));

System.out.println("SunJCE: " + Arrays.toString(resultSun));

if (Arrays.equals(resultBC, resultSun)) {

System.out.println("OK: Signatures are equal!");

} else {

System.err.println("FAIL: Signatures are not equal!");

}

}

}