Re: NSS - TLS Session Hash and Extended Master Secret doesn't work in FIPS mode with Java 1.8.0_161

Doing more reading on Extended Master Secret extension support in NSS, I understand that support is "off" by default. But I can't see in any documentation how to "turn it on". Any help out there?

On Monday, February 26, 2018 at 8:57:16 AM UTC-5, James Covington wrote:
> Not sure if this post belongs here or in mozilla.dev.tech.crypto, have posted to both:
> 
> I am using nss 3.23.0.0 with nspr 4.12.0.0 in FIPS mode to provide the cryptography for my Red Hat JBoss EAP 6.4.14 application server hosted on Windows 2008 R2 server. After updating my JDK version from 1.8.0_152 to 1.8.0_162, I received this exception during TLS handshake in my server.log:
> 
>   RSA master secret generation error:
>  java.security.InvalidAlgorithmParameterException: Key format must be RAW
>  	at com.sun.crypto.provider.TlsMasterSecretGenerator.engineInit(TlsMasterSecretGenerator.java:67)
>  	at javax.crypto.KeyGenerator.init(KeyGenerator.java:454)
>  	at javax.crypto.KeyGenerator.init(KeyGenerator.java:430)
>  	at sun.security.ssl.Handshaker.calculateMasterSecret(Handshaker.java:1261)
>  	at sun.security.ssl.Handshaker.calculateKeys(Handshaker.java:1183)
>  	at sun.security.ssl.ServerHandshaker.processMessage(ServerHandshaker.java:301)
>  	at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1052)
>  	at sun.security.ssl.Handshaker.process_record(Handshaker.java:987)
>  	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1072)
>  	at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1385)
>  	at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1413)
>  	at sun.security.ssl.SSLSocketImpl.getSession(SSLSocketImpl.java:2306)
>  	at org.apache.tomcat.util.net.jsse.JSSESocketFactory.handshake(JSSESocketFactory.java:201)
>  	at org.apache.tomcat.util.net.JIoEndpoint.setSocketOptions(JIoEndpoint.java:1114)
>  	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:926)
>  	at java.lang.Thread.run(Thread.java:748)
> 
> In reviewing Oracle's release notes, I found that one of the new features effective in build 161 is the addition of TLS Session Hash and Extended Master Secret extension support in the jsse provider. By following the guidance in the release note, and adding -Djdk.tls.UseExtendedMasterSecret=false to my JAVA_OPTS variable, the exception is no longer raised.
> 
> I have read that NSS supports Extended Master Secret extension effective with version 3.21, so I'm not sure where to direct a bug report. Perhaps this extension is not supported in FIPS mode?