Re: Need help with James 3.4, smtpserver, and keystore file.

Thanks Garry for your reply, and I am sorry for the slowness of my
response, I been busy doing internet research trying to grok everything
you said in your suggestions. I think the best thing to do is to walk
you and other readers through the steps I took to create the LetsEncrypt
certificate and put it in the keystore. Then I will comment on each of
your suggestions and await for further clarification on what I should
try/do next.  I am not at all familiar with the code and inner workings
of Apache James so to me a lot of this is like trying to understand
what  is in a room by looking through a keyhole! LOL

OK, these are the steps I took -

To create the LetsEncrypt certificate I used the DNS challenge method to
create a wildcard certificate for all the domains and subdomains that I
provide email services for. This was done as follows using the certbot
command -

certbot certonly  --config-dir /etc/letsencrypt_forApacheJames
--dns-rfc2136 --dns-rfc2136-credentials
/etc/letsencrypt_forApacheJames/james/rfc2136.ini
--dns-rfc2136-propagation-seconds 10 --server
https://acme-v02.api.letsencrypt.org/directory
--preferred-challenges=dns --email [email protected] --agree-tos
-d domainname1.com -d *.domainname1.com  -d domainname2.com -d
*.domainname2.com   /etc.../

If I understand things correctly this produced 4 certificate and chain
files -

cert.pem is the certificate containing the public key for my domains.
chain.pem is the certificate for my certificate authority - LetsEncrypt
fullchain.pem is also created by certbot = cert.pem + chain.pem
privkey.pem containing the private key for my domains.

Next I created an empty keystore in the conf directory for Apache James -

cd /mail/apache-james-3.4/james-server-app-3.4.0/conf
mkdir keystore.privateFiles
cd keystore.privateFiles
keytool -genkeypair -keyalg RSA -alias emptykeystore -keystore keystore.jks

and I  migrated the keystore to PKCS12 format and deleted the
emptykeystore alias using

keytool -importkeystore -srckeystore keystore.jks -destkeystore
keystore.jks -deststoretype pkcs12
keytool -delete -alias emptykeystore -keystore keystore.jks

Next I converted the keys created by certbot into the proper format for
importing it into the keystore, using openssl to convert the keys -

openssl pkcs12 -export -in fullchain.pem -inkey privkey.pem -out
pkcs.p12 -name james

This creates a file called pkcs.p12 which I then import into the
keystore file -

keytool -importkeystore -deststorepass mypassword -destkeypass
mypassword -destkeystore keystore.jks -srckeystore
/etc/letsencrypt_forApacheJames/live/mydomain.com-0001/pkcs.p12
-srcstoretype PKCS12 -srcstorepass mypassword -alias james

I left out a few details like sensitive info, cd'ing to various
different directories and setting up a link to the keystore file. The
passwords are simple alphanumeric character strings. I will intersperse
the rest of my reply in-between your comments -

On 1/3/20 2:12 PM, Garry Hurley wrote:
> Okay. One thing I noticed before. The key took command will hash the password with the appropriate algorithm and use it to access the keystone file. James code is not guaranteed to use the appropriate hashing algorithm. Try the following:
I am not sure what you mean and are referring to as the hashing
algorithm though I do know what hashing is. In the steps I showed you,
that I took to create the keystore, I see parameters that specify
encryption algorithms but nothing that specifies a hashing algorithm.
> 1 sift through the code, find the hashing algorithm used by James and hard code the hashed key in the configuration file. 
Oh boy, I am not set up to build James from source and suspect that will
be a steep learning curve. I will table this option for the moment and
consider it if option 2 doesn't work.
> 2, find out which hashing algorithm is used to encrypt the key and specify it in the configuration file
This sounds promising except I don't know what hashing algorithm was
used in creating the keys. I suspect you are referring to the encryption
algorithm? (I know nothing about how the encryption is actually done,
but could imagine that it is based on a hash of some kind. I believe
that I am using the RSA encryption algorithm but check me on that... 
Also I don't know how to specify the "hashing algorithm" in the
configuration files so an example would be very helpful.
> 3, create a new key store WITHOUT a password and use that one instead. 
This sounds dangerous and I will try this approach also if all else
fails... But yeah it sounds like another path I could follow...
> For ease of use, #3 is the clear winner. For maximum security, #1 would be preferred.   I am almost certain you will find it is due to a hashing algorithm or, alternatively, a character in the password that is expressly probibited in an XML file (like an ampersand or greater/less than sign for example). 
The password is only composed of lower case letters and numbers. No
punctuation characters are used.

I don't know if this is applicable but in my internet searches looking
for discussions about the same or similar error messages that I am
seeing, I am seeing some references to problems with the tools from
BouncyCastle.  Here are a couple of links, perhaps you or some other
guru could check these out and see if I am perhaps experiencing the same
or similar issue as these folks are talking about -

https://stackoverflow.com/questions/53542198/adding-bouncycastle-provider-breaks-keystore-load

https://github.com/bcgit/bc-java/issues/586

This is above my pay grade to understand so again many thanks and I
appreciate yours and any other help offered...    Marc...

>
> Sent from my iPhone
>
>> On Dec 28, 2019, at 9:35 PM, Marc Chamberlin <[email protected]> wrote:
>>
>> Hello again,  It appears that I have managed to make a bit of progress
>> on my own since my original post. For some odd reason I got past the
>> FileNotFound exception by changing (adding) execute permission to the
>> directory containing the keystore file. So now it is set as follows -
>>
>> quasar:/mail/apache-james-3.4/james-server-app-3.4.0/conf # ll -d
>> keystore.privateFiles
>> drwxrwxrwx 2 james mail 4096 Dec 27 22:11 keystore.privateFiles
>>
>> It is a puzzler (at least to me) why adding execute permissions would
>> allow James to find my Keystore file. But I immediately hit another
>> snag, James is not correctly applying the password to access the
>> keystore file and throwing another exception. The smtpserver.xml is
>> configured as follows -
>>
>>      <tls socketTLS="false" startTLS="true">
>>       
>> <keystore>file:/mail/apache-james-3.4/james-server-app-3.4.0/conf/keystore</keystore>
>>        <secret>mypassword</secret>
>>       
>> <provider>org.bouncycastle.jce.provider.BouncyCastleProvider</provider>
>>        <algorithm>SunX509</algorithm>
>>      </tls>
>>
>> To test the keystore file I executed the following command and it worked
>> fine -
>>
>> quasar:/mail/apache-james-3.4/james-server-app-3.4.0/conf # keytool
>> -list -v -keystore
>> /mail/apache-james-3.4/james-server-app-3.4.0/conf/keystore -storetype
>> PKCS12 -storepass mypassword
>>
>> The exception and stack walkback I am getting from James is shown
>> below.  I am continuing to Google for answers but no joy so far...
>> Anyone here got any ideas? Thanks again in advance...     Marc.
>>
>> ----
>>
>> INFO   | jvm 1    | 2019/12/28 17:13:27 | WARN  17:13:26,963 |
>> org.apache.james.container.spring.context.JamesServerApplicationContext
>> | Exception encountered during context initialization - cancelling
>> refresh attempt:
>> org.springframework.beans.factory.BeanCreationException: Error creating
>> bean with name 'smtpserver': Invocation of init method failed; nested
>> exception is java.io.IOException: keystore password was incorrect
>> INFO   | jvm 1    | 2019/12/28 17:13:27 | INFO  17:13:26,964 |
>> org.apache.james.mailetcontainer.impl.JamesMailSpooler | start dispose() ...
>> INFO   | jvm 1    | 2019/12/28 17:13:27 | INFO  17:13:26,964 |
>> org.apache.james.mailetcontainer.impl.JamesMailSpooler | thread shutdown
>> completed.
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |
>> INFO   | jvm 1    | 2019/12/28 17:13:27 | WrapperSimpleApp: Encountered
>> an error running main:
>> org.springframework.beans.factory.BeanCreationException: Error creating
>> bean with name 'smtpserver': Invocation of init method failed; nested
>> exception is java.io.IOException: keystore password was incorrect
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |
>> org.springframework.beans.factory.BeanCreationException: Error creating
>> bean with name 'smtpserver': Invocation of init method failed; nested
>> exception is java.io.IOException: keystore password was incorrect
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |       at
>> org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:133)
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |       at
>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:396)
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |       at
>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1507)
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |       at
>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:519)
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |       at
>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |       at
>> org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:293)
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |       at
>> org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:223)
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |       at
>> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:290)
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |       at
>> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:191)
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |       at
>> org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:638)
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |       at
>> org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:942)
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |       at
>> org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482)
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |       at
>> org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:139)
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |       at
>> org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:93)
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |       at
>> org.apache.james.container.spring.context.JamesServerApplicationContext.<init>(JamesServerApplicationContext.java:40)
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |       at
>> org.apache.james.app.spring.JamesAppSpringMain.init(JamesAppSpringMain.java:56)
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |       at
>> org.apache.james.app.spring.JamesAppSpringMain.main(JamesAppSpringMain.java:42)
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |       at
>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |       at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |       at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |       at
>> java.lang.reflect.Method.invoke(Method.java:498)
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |       at
>> org.tanukisoftware.wrapper.WrapperSimpleApp.run(WrapperSimpleApp.java:240)
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |       at
>> java.lang.Thread.run(Thread.java:748)
>> INFO   | jvm 1    | 2019/12/28 17:13:27 | Caused by:
>> java.io.IOException: keystore password was incorrect
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |       at
>> sun.security.pkcs12.PKCS12KeyStore.engineLoad(PKCS12KeyStore.java:2059)
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |       at
>> sun.security.provider.KeyStoreDelegator.engineLoad(KeyStoreDelegator.java:238)
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |       at
>> sun.security.provider.JavaKeyStore$DualFormatJKS.engineLoad(JavaKeyStore.java:70)
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |       at
>> java.security.KeyStore.load(KeyStore.java:1445)
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |       at
>> org.apache.james.protocols.lib.netty.AbstractConfigurableAsyncServer.buildSSLContext(AbstractConfigurableAsyncServer.java:405)
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |       at
>> org.apache.james.protocols.lib.netty.AbstractConfigurableAsyncServer.init(AbstractConfigurableAsyncServer.java:263)
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |       at
>> org.apache.james.protocols.lib.netty.AbstractServerFactory.init(AbstractServerFactory.java:57)
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |       at
>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |       at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |       at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |       at
>> java.lang.reflect.Method.invoke(Method.java:498)
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |       at
>> org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:344)
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |       at
>> org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:295)
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |       at
>> org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:130)
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |       ... 22 more
>> INFO   | jvm 1    | 2019/12/28 17:13:27 | Caused by:
>> java.security.UnrecoverableKeyException: failed to decrypt safe contents
>> entry: java.lang.IllegalStateException: password has been cleared
>> INFO   | jvm 1    | 2019/12/28 17:13:27 |       ... 36 more
>>
>> -- 
>>
>>  --...  ...--  .----.  ...    -..  .    .--  .-  --...  .--.  -..-  .--     --  .-  .-.  -.-. 
>>
>>
>> *Computers: the final frontier. These are the voyages of the user Marc.
>> His mission: to explore strange new hardware. To seek out new software
>> and new applications.
>> To boldly go where no Marc has gone before!
>> *
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
>

-- 

  --...  ...--  .----.  ...    -..  .    .--  .-  --...  .--.  -..-  .--     --  .-  .-.  -.-. 


*Computers: the final frontier. These are the voyages of the user Marc.
His mission: to explore strange new hardware. To seek out new software
and new applications.
To boldly go where no Marc has gone before!
*