Re: James 3.8 one OpenSuSE 15.5

Hi Matt,

Let me answer you this time below in each of your points :)

Cheers,

Rene.

On 1/9/24 21:27, cryptearth wrote:
> Hello Rene,
>
> thank you for the kind words.
>
> May let me answer your points:
>
> - Spring vs. Guice: I use James since the Beta of 3.0 and if I recall 
> correctly back then the Spring package was the one to use. By the 
> usual "never change a running system" I not changed since. I had a 
> quick look into it, but unfortunately the links on the install 
> instructions https://james.apache.org/server/install.html only lead to 
> a 404 - so there's something missing. Also I seem to require docker - 
> which I don't have any experience with. I'd rather prefer just to 
> build a package, unpack it and start it without having to deal with 
> any extra containerization around it. It seems I require some help to 
> setup a guice install.

Regarding the 404 on the install page, it looks like you are right. It's 
not normal and I might take a look at it, thanks for the feedback! 
However, docker is one way, but like Sprint you can still just run a jar 
with Guice as well. You can see the download page for example (where the 
links seem to work) : https://james.apache.org/download.cgi .

If you want to try to switch to Guice, the JPA james server is the one 
for you, that you can plug with MariaDB. The distributed version is for 
heavy deployments (which I don't think interests you here) with 
cassandra, opensearch, rabbitmq, ... => 
https://www.apache.org/dyn/closer.lua/james/server/3.8.0/james-server-jpa-guice.zip 
. You will see in the README as well in the zip some basic instructions 
to run it, with docker or manually with the jar like you are used to.

I totally understand the reason behind the fact that you started with 
Spring, so far it works, and it's safe to stick to it. Quite a fair 
statement actually. The problem is that though the main development 
efforts since a while are being done on Guice, and the Spring one is 
quickly maintained, as nobody seems fully willing to keep developing on 
it now. It works but could have potentially some vulnerabilities... I'm 
thinking for example the spring dependencies, that are old and outdated 
I think^^'

>
> - as for using MariaDB: This goes back to 2015 when I started to rent 
> my server and own domain. Back then I had basically no knowledge about 
> server administration let alone set up a mail server. Although 
> there're complete guides on how to setup postfix with dovecot and such 
> on opensuse I not managed to get it working. So I looked for a simple 
> complete solution and ended up using Citadel/UX. For some reason with 
> some update it broke to a state I wasn't able to start it again. Hence 
> I wasn't able to get any data out of it as for some reason I just was 
> not able to access the BerkeleyDB data file. As I also wasn't able to 
> get the regular stuff with maildir running I just wanted a setup using 
> a regular MySQL not just for config- and user-data but also as the 
> mail storage backend so that in the event I encountered such a 
> situation again I can just use regular tools to restore a mysql 
> database. As I'm also a hobbyist Java developer for me it was a simple 
> idea of "there has to be some simple mail server implemented in Java" 
> - and this is how I ended up discovering and using James ever since.
>
> It also already came into play that for some reason I had to manually 
> restore my database - exactly the reason why I chose MySQL as storage 
> backend. As it was helpful for the reason I chose it I rather stick to 
> it for that reason. The switch from MySQL to MariaDB was just 
> something OpenSuSE did - so I followed it. As a user I not noticed any 
> differences.
>
> As for the database issues mentioned: For some reason I had to set up 
> the MySQL- and later the MariaDB-repos instead of using what comes 
> with the OpenSuSE repos. I don't know what the difference was but it 
> prevented the initial setup of the database structure on first run. 
> Hence I'm glad that this is now fixed - whoever is reponsible for it.
I get this :) I was just trying to inform you that these days we work 
also on a Postgresql exclusive implementation for James. JPA is good to 
be able to plug to a lot of different SQL databases with the same code, 
but performances can be a bit meh. We are working on a full postgresql 
implementation of James in parallel as most people using JPA in the 
community are using Postgresql as their database (as it is quite a 
popular one). The goal is to have better performance here than with JPA 
for this particular db. But likely JPA might be enough for your singular 
usage. Was just throwing that here if maybe it was interesting for you.
>
>
> - sync between main server and backup: I guess the additional forward 
> from the main server to the backup and handle them independently seems 
> like a neat idea simple to implement. I mostly rely on my backup when 
> doing maintenance on the root like OS updates which require a system 
> reboot. It's not I want to try to build a distributed synchronized 
> cluster. I guess for that there're better solutions out there. Looking 
> thru the docs there's something pointing to 
> https://github.com/apache/james-project/blob/james-project-3.8.0/docs/modules/servers/pages/distributed/run-docker.adoc 
> - but that doesn't exists or isn't set to public, same as for the 
> guice setup. So it seems there's documentation for what I want to do - 
> but it's not available to me.

That new Antora documentation is still incomplete and yeah there is 
issues for some reasons with the distributed part that does not render 
each time during build... Might need tot ake an other look at it! For 
the temporary link of it though if it interests you: 
https://james.staged.apache.org/james-project/3.8.0/index.html

Distributed doc in the source code though if you feel like building it 
yourself: 
https://github.com/apache/james-project/tree/master/server/apps/distributed-app/docs

I think yes forwarding your mails to both your main and backup servers 
is simple and efficient :)

>
> Have a good one.
>
> Matt
>
> On 09.01.24 08:44, Rene Cordier wrote:
>> Hello,
>>
>> Interesting read thanks.
>>
>> Few remarks though, but feel free to agree or not :) :
>>
>> - any reason to still run the spring version of James? That version I 
>> believe is not always up-to-date with dependencies (specially the 
>> sprint dependencies). Truth is it might get deprecated and removed in 
>> later release if a consensus is reached, as all active dev is done on 
>> the Guice versions now. You might want to check that (JPA Guice here 
>> probably closest to the Spring version)
>>
>> - note that there is active development at the moment (still WIP) 
>> regarding a Postgresql reactive implementation with james (Guice). I 
>> saw you seem to use MariaDB but might interest you for the future maybe?
>>
>> - for sync between your main server and backup, maybe just redirect 
>> the mails to both platforms, and let them treat the mails separately? 
>> If not I guess db backup and sync between the two could do it too.
>>
>> Good job on running latest James on the new OpenSuse though, thanks 
>> for sharing!
>>
>> Rene.
>>
>> On 1/9/24 10:17, cryptearth wrote:
>>> Hello there everybody,
>>>
>>> happy new year 2024 to all of you.
>>> It's been quite some time since I was last active on this maillist. 
>>> After
>>> upgrading my server to OpenSuSE 15.5 and James to 3.8 and after 
>>> reading the
>>> recent topics on the web archive I decided to post a small round up 
>>> how it
>>> went this way.
>>>
>>> Important word of warning and security:
>>> This is only a personal write-down how I did my setup. I'm aware 
>>> there're
>>> still some security improvements to be done. This is meant as an 
>>> informative
>>> guide. I'm by no means responsible for any security issues that may 
>>> arise by
>>> following this. I myself got hacked by the Log4J issue mostly due to 
>>> not
>>> applying the available update fast enough. It's your own 
>>> responsibility to
>>> make your setup secure. All I can offer are some guidelines from 
>>> personal
>>> experience.
>>>
>>> That warning out now let's get to the real topic.
>>>
>>> As several of the active devs here know I use James for quite some 
>>> time now
>>> and over the years encountered several issues mostly specific to 
>>> OpenSuSE and
>>> the packages provided by its repositories. Luckily this time 
>>> everything went
>>> smoothly.
>>>
>>> I started from a fresh clean "server"-type install, but this should be
>>> applicable to any existing system as well. So the first step is to 
>>> fully update
>>> the new system.
>>>
>>> As OpenSuSE comes with postfix I replace it with sendmail for later. 
>>> I guess
>>> it's also possible with postfix or exim or any other MTA providing the
>>> required sendmail dependency. Using sendmail was just the simplest 
>>> option I
>>> figured out when I first used it.
>>> The keyword here is "nullclient": It configures sendmail to just 
>>> forward
>>> everything to James without any handling itself. For this sendmail 
>>> comes with
>>> an almost ready-to-use config which only needs one change in 
>>> /etc/sendmail/
>>> linux.nullclient.mc in line 27 by replacing the example
>>> "mailhub.domain.notused" with "localhost". Then just generate the new
>>> sendmail.cf from it (either as root or via sudo):
>>> m4 /etc/mail/linux.nullclient.mc > /etc/sendmail.cf
>>>
>>> I also have to edit the systemd script. For some reason the 
>>> systemctl edit
>>> command works a bit unexpected for simple edits so by appending 
>>> --full it
>>> becomes a full override. The service file needs two changes:
>>>
>>> 1) moving the PIDFile from /var/run to /run as hinted by systemctl 
>>> status
>>> message:
>>> PIDFile=/run/sendmail.pid
>>>
>>> 2) remove the -bd flag from the start args to prevent sendmail from 
>>> blocking
>>> port TCP/25:
>>> Environment="SENDMAIL_ARGS=-L -Am -q30m -om"
>>>
>>> Check if sendmail service is enabled and started: systemctl status 
>>> sendmail
>>> At least enabled it so it gets started at reboot. As James isn't 
>>> setup yet to
>>> take its mails don't start it yet - it could cause a hang into an 
>>> infinite
>>> loop.
>>>
>>> Before get to James I prepare the rest of the system by install 
>>> these packages
>>> (can be installed along when switching postfix for sendmail):
>>> - java-17-openjdk-devel
>>> - git
>>> - maven
>>> - mariadb
>>> - phpMyAdmin-apache
>>> - yast2-http-server
>>> The required dependencies get pulled in by the resolver. After 
>>> installing the
>>> packages YaST tells you to secure MariaDB. So I do this right away 
>>> by running
>>> mariadb-secure-installation as root/sudo and set a root password. 
>>> All other
>>> questions can be answered with thier default option (just hit enter).
>>>
>>> To get phpMyAdmin running only the initial setup using "yast 
>>> http-server" is
>>> required. Select enable PHP in the 2nd step and complete the setup 
>>> by just
>>> hitting F10 a couple times. If the Apache2 server is accessible via the
>>> internet care should be taken by limiting phpMyAdmin to localhost. 
>>> Edit /etc/
>>> apache2/conf.d/phpMyAdmin.conf line 27 to "Require local" and use 
>>> ssh tunnels.
>>> If you have services like a vpn or a tor hidden service running you 
>>> may want
>>> to secure it even further with something like an .htaccess or 
>>> similar. This
>>> way you prevent attackers from trying to bruteforce your database 
>>> password.
>>>
>>> Complete the pre-setup by use phpMyAdmin to add a new user for James.
>>>
>>> Now let's build James:
>>> git clone https://github.com/apache/james-project -b 
>>> james-project-3.8.0
>>> cd james-project
>>> mvn clean package -DskipTests -T 1C
>>> As I use the spring package I then unpack the tar in 
>>> james-project/server/
>>> apps/spring-app/target/james-server-spring-app-3.8.0-app.tar.gz.
>>> I also need to get the mariadb-connector/J: 
>>> https://mariadb.com/downloads/
>>> connectors/ and place the .jar in the james-server-spring-app-3.8.0/lib
>>> folder.
>>> I'm not sure why this isn't distributed along or part of the package 
>>> manifest
>>> to pull from the repos - guess this is some licence nonesense.
>>>
>>> It's config time now - and there's not that much to do.
>>> As I don't use POP3 the first I do is disable the POP3 server.
>>> Next I set up the database stuff by change the driver to
>>> org.mariadb.jdbc.Driver
>>> and the connector-string to
>>> jdbc:mariadb://localhost:3306/james
>>> Btw: As pretty much any modern JDBC driver should be a type 4.x 
>>> there's no
>>> need to specify the driver class name or do something like
>>> Class.forName()
>>> As long as the driver jar is in classpath on startup of the JVM it 
>>> gets pulled
>>> in via the ServiceLoader-API.
>>>
>>> Next I set my domain and the greetings in the imap and smtp servers. 
>>> I also
>>> enable StartTLS for both imap and smtp. As for outgoing smtp I also 
>>> enable
>>> StartTLS by adding this to the RemoteDelievery in the 
>>> mailetcontainer.xml:
>>> <startTLS>true</startTLS>
>>> This should be added by default as it's only mentioned in the 
>>> RemoteDelievery
>>> javadoc but not in the XML doc nor in the config doc.
>>> Until now I used a Java keystore for the certificate and key. But 
>>> using the new
>>> PEM files directly skips this. Thanks for who added this.
>>>
>>> The last part is to deal with sendmail and apache2/php mail. For 
>>> this I first
>>> add two users to James (along with others like postmaster): 
>>> webmaster and
>>> local-service. Then I add this to the mailetcontainer.xml right at 
>>> the top of
>>> the root processor:
>>>
>>>          <matcher name="local-service"
>>> match="org.apache.james.mailetcontainer.impl.matchers.And">
>>>                  <matcher match="RemoteAddrInNetwork=127.0.0.0/8" />
>>>                  <matcher
>>> match="org.apache.james.mailetcontainer.impl.matchers.Not">
>>>                          <matcher match="SMTPAuthSuccessful" />
>>>                  </matcher>
>>>                  <matcher
>>> match="org.apache.james.mailetcontainer.impl.matchers.Not">
>>>                          <matcher 
>>> match="[email protected]" />
>>>                  </matcher>
>>>                  <matcher
>>> match="org.apache.james.mailetcontainer.impl.matchers.Not">
>>>                          <matcher match="RecipientIs=local-
>>> [email protected]" />
>>>                  </matcher>
>>>          </matcher>
>>>
>>>          <mailet match="local-service" class="Redirect">
>>> <to>[email protected]</to>
>>>          </mailet>
>>>
>>> To explain its purpose:
>>> When starting James via crontab by root crond sends a start report via
>>> sendmail. In order for sendmail to not get stuck in an endless loop 
>>> trying to
>>> deliver the crond status I have to provide some drop off destination 
>>> for it.
>>> But as I also want to send e-mails by PHP I had to get a bit creative
>>> filtering. It basically goes like this:
>>>
>>> IF incoming mail comes from localhost
>>> AND is NOT authenticated via smtp auth
>>> AND is NOT sent by webmaster (from php)
>>> AND is NOT already redirected to local-service
>>> THEN redirect to local-service
>>>
>>> This rule maybe can be written better - but it was the simplest I 
>>> came up
>>> with. The check for the recipient is required so the mail doesn't get
>>> redirected to itself again. The check for the webmaster is required as
>>> sendmail drops off php mails with wwwrun@localhost, which is 
>>> overriden by the -
>>> f parameter to webmaster. Any other client that regular logs in via 
>>> AUTH is
>>> ignored anyway. And the whole thing is limited to localhost.
>>>
>>> Finally, add a
>>> @reboot /path/to/james/bin/james start
>>> to roots crontab for james autostart.
>>>
>>> If everything was done correctly james should start up just fine for 
>>> the first
>>> time (it actually already had to for adding the users - but this can 
>>> be done
>>> now).
>>> I do the first start via "console" to verify everything is done 
>>> correctly.
>>> After all is done and james starts correctly I just reboot the 
>>> system. Now
>>> when all comes up by itself the local-service user should have a 
>>> mail confirm
>>> the successful start of James which also confirms that the sendmail 
>>> nullclient
>>> and the mailet redirect works.
>>>
>>> A common failure can be an issue with name resolution. In this case 
>>> you have
>>> to manually set /etc/hosts and reboot again. This can also be done 
>>> early on as
>>> one of the first few steps after updating the system.
>>>
>>> To send mails from apache this can be done like this:
>>>
>>> <?php
>>>          $to="[email protected]";
>>>          $subject=$content="this is a apache webmail test";
>>>          $headers = array(
>>>                  'From' => 'CryptEarth.de Webmaster 
>>> <[email protected]>',
>>>                  'Reply-To' => '[email protected]',
>>>                  'X-Mailer' => 'PHP/' . phpversion()
>>>          );
>>>          mail($to, $subject, $content, $headers, 
>>> "[email protected]");
>>> ?>
>>>
>>> A few improvements that can (and should) be done:
>>>
>>> - don't start james as root but rather move the ports to 8025 and 
>>> 8143 and use
>>> a firewall rule to redirect 25 to 8025 and 143 to 8143 ... but I 
>>> have to figure
>>> that out
>>> - maybe rethink about sendmail nullclient and do similar with 
>>> postfix or exim
>>> - maybe redo the mailet redirect for local-service and php-mail
>>> - setting up DKIM on outgoing mails
>>>
>>> Currently I have James running on two servers: my main root as 
>>> primary and on
>>> a small sbc at home as a backup. Is there a way to synchronize the 
>>> two servers
>>> on the application level. Or would it be enough to just synchronize 
>>> the two
>>> datases secured by a ssh tunnel? How to handle the differences in 
>>> the counters
>>> when both servers receive an e-mail each which have to get 
>>> synchronized in
>>> both ways?
>>>
>>> Have a nice week.
>>> Greetings from Germany
>>>
>>> Matt
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [email protected]
>>> For additional commands, e-mail: [email protected]
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [email protected]
>> For additional commands, e-mail: [email protected]
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
>
>