Re: James 3.8 one OpenSuSE 15.5
cryptearth <[email protected]> Sun, 14 Jan 2024 03:07:53 +0100
| Newsgroups | gmane.comp.jakarta.james.user |
|---|---|
| Message-ID | <[email protected]> |
Hi everyone, so, I had a look into Guice+JPA and it's quite easy to setup by just reuse the configs I have for the Spring build. And although I still can use james-cli (although a bit different) I now have access to the REST-api via the webadmin (until now it seems I misunderstood it: I thought the "web admin" to be some browser admin panel instead of just an REST-api). And although it seems rather easy to use it to do the same as with the james-cli and even more - looking in the list archive: Is there any "client" for it like some webpage or some standalone gui? Guess if not it'd be a nice project to get something together - although I'm only a hobbyist and hence the "most advanced" I was able to get done was a simple networked battle ships game with some very ugly ui-code for the placement of the ships. So I guess if at all I can only provide some simple stuff and far from the code standards of ASF. The next will now be to switch over to Guice+JPA and also upgrade my backup server. Nice to see I still can learn about James - just another confirmation it was the right choice after Citadel/UX got broken, although I'll re-visit other setups with sendmail, postfix or exim just for the fun of additional knowledge. So long ... have a good one. Matt Am 11.01.24 um 05:40 schrieb Rene Cordier: > Hello Matt, > > Thanks for keeping reporting dead links, I try to fix them every time > firing PRs :) > > Regarding the correct link to the jpa guice sample conf: > https://github.com/apache/james-project/tree/master/server/apps/jpa-app/sample-configuration > > Regarding the link in a previous email with jpa-guice installation > guide (but I think the readme is likely more complete): > https://james.apache.org/server/install/guice-jpa.html > > For this though: > https://james.staged.apache.org/james-project/3.8.0/servers/basic/conf/index.html, > yes that's still not done, it's normal atm, as said this new doc is > still a work in progress ^^' > > Regarding your mail with INVALID suffixed to it... No idea really^^' > Might want to ask the people managing the mailing lists I guess. > > Well if anything else don't hesitate :) > > Rene. > > On 1/11/24 02:45, cryptearth wrote: >> Hello Rene, >> >> thank you for your reply. >> >> I tried to look into the docs and the readme, but I get redirected to >> these two pages without content: >> >> https://github.com/apache/james-project/tree/master/server/container/guice/jpa-guice/sample-configuration >> >> https://james.staged.apache.org/james-project/3.8.0/servers/basic/conf/index.html >> >> >> From the readme it looks simple to get the jpa-guice package to run >> as it seems I can re-use a lot of my current configs. As for the >> required keystore: All I found was its use for TLS - as I use the new >> PEM variant it should also work this way. I'll give it a shot in a VM. >> >> Nice to see that the pop3 server is disabled by default. I'm sure >> this protocol still has its uses, but as it downloads the messages >> from the server and removes them while doing so I only see an >> application where messages not supposed to be left on the server like >> for quota reasons or something like privacy. >> >> As for MySQL/MariaDB vs PostgreSQL: The only think I know about >> PostgreSQL is that it exist but never used it due to most beginner >> books I read when I started roughly over 15 years ago all mentioned >> it but only used MySQL anyways. Shouldn't make a difference which to >> use. >> >> As for forwarding mails to my backup: I already came up with some >> ideas to make this work in both ways without causing loops so that >> mails received by the backup will also get forwarded to the main when >> it comes back up. >> >> btw: I noticed that my domain gets an .INVALID appended to it - but >> only here on this mailing list. Testmails to other services don't >> show this. Anyone maybe has an idea about that? >> >> >> Have a good one. >> >> Matt >> >> On 10.01.24 04:28, Rene Cordier wrote: >>> Hi Matt, >>> >>> Let me answer you this time below in each of your points :) >>> >>> Cheers, >>> >>> Rene. >>> >>> On 1/9/24 21:27, cryptearth wrote: >>>> Hello Rene, >>>> >>>> thank you for the kind words. >>>> >>>> May let me answer your points: >>>> >>>> - Spring vs. Guice: I use James since the Beta of 3.0 and if I >>>> recall correctly back then the Spring package was the one to use. >>>> By the usual "never change a running system" I not changed since. I >>>> had a quick look into it, but unfortunately the links on the >>>> install instructions https://james.apache.org/server/install.html >>>> only lead to a 404 - so there's something missing. Also I seem to >>>> require docker - which I don't have any experience with. I'd rather >>>> prefer just to build a package, unpack it and start it without >>>> having to deal with any extra containerization around it. It seems >>>> I require some help to setup a guice install. >>> >>> Regarding the 404 on the install page, it looks like you are right. >>> It's not normal and I might take a look at it, thanks for the >>> feedback! However, docker is one way, but like Sprint you can still >>> just run a jar with Guice as well. You can see the download page for >>> example (where the links seem to work) : >>> https://james.apache.org/download.cgi . >>> >>> If you want to try to switch to Guice, the JPA james server is the >>> one for you, that you can plug with MariaDB. The distributed version >>> is for heavy deployments (which I don't think interests you here) >>> with cassandra, opensearch, rabbitmq, ... => >>> https://www.apache.org/dyn/closer.lua/james/server/3.8.0/james-server-jpa-guice.zip >>> . You will see in the README as well in the zip some basic >>> instructions to run it, with docker or manually with the jar like >>> you are used to. >>> >>> I totally understand the reason behind the fact that you started >>> with Spring, so far it works, and it's safe to stick to it. Quite a >>> fair statement actually. The problem is that though the main >>> development efforts since a while are being done on Guice, and the >>> Spring one is quickly maintained, as nobody seems fully willing to >>> keep developing on it now. It works but could have potentially some >>> vulnerabilities... I'm thinking for example the spring dependencies, >>> that are old and outdated I think^^' >>> >>>> >>>> - as for using MariaDB: This goes back to 2015 when I started to >>>> rent my server and own domain. Back then I had basically no >>>> knowledge about server administration let alone set up a mail >>>> server. Although there're complete guides on how to setup postfix >>>> with dovecot and such on opensuse I not managed to get it working. >>>> So I looked for a simple complete solution and ended up using >>>> Citadel/UX. For some reason with some update it broke to a state I >>>> wasn't able to start it again. Hence I wasn't able to get any data >>>> out of it as for some reason I just was not able to access the >>>> BerkeleyDB data file. As I also wasn't able to get the regular >>>> stuff with maildir running I just wanted a setup using a regular >>>> MySQL not just for config- and user-data but also as the mail >>>> storage backend so that in the event I encountered such a situation >>>> again I can just use regular tools to restore a mysql database. As >>>> I'm also a hobbyist Java developer for me it was a simple idea of >>>> "there has to be some simple mail server implemented in Java" - and >>>> this is how I ended up discovering and using James ever since. >>>> >>>> It also already came into play that for some reason I had to >>>> manually restore my database - exactly the reason why I chose MySQL >>>> as storage backend. As it was helpful for the reason I chose it I >>>> rather stick to it for that reason. The switch from MySQL to >>>> MariaDB was just something OpenSuSE did - so I followed it. As a >>>> user I not noticed any differences. >>>> >>>> As for the database issues mentioned: For some reason I had to set >>>> up the MySQL- and later the MariaDB-repos instead of using what >>>> comes with the OpenSuSE repos. I don't know what the difference was >>>> but it prevented the initial setup of the database structure on >>>> first run. Hence I'm glad that this is now fixed - whoever is >>>> reponsible for it. >>> I get this :) I was just trying to inform you that these days we >>> work also on a Postgresql exclusive implementation for James. JPA is >>> good to be able to plug to a lot of different SQL databases with the >>> same code, but performances can be a bit meh. We are working on a >>> full postgresql implementation of James in parallel as most people >>> using JPA in the community are using Postgresql as their database >>> (as it is quite a popular one). The goal is to have better >>> performance here than with JPA for this particular db. But likely >>> JPA might be enough for your singular usage. Was just throwing that >>> here if maybe it was interesting for you. >>>> >>>> >>>> - sync between main server and backup: I guess the additional >>>> forward from the main server to the backup and handle them >>>> independently seems like a neat idea simple to implement. I mostly >>>> rely on my backup when doing maintenance on the root like OS >>>> updates which require a system reboot. It's not I want to try to >>>> build a distributed synchronized cluster. I guess for that there're >>>> better solutions out there. Looking thru the docs there's something >>>> pointing to >>>> https://github.com/apache/james-project/blob/james-project-3.8.0/docs/modules/servers/pages/distributed/run-docker.adoc >>>> - but that doesn't exists or isn't set to public, same as for the >>>> guice setup. So it seems there's documentation for what I want to >>>> do - but it's not available to me. >>> >>> That new Antora documentation is still incomplete and yeah there is >>> issues for some reasons with the distributed part that does not >>> render each time during build... Might need tot ake an other look at >>> it! For the temporary link of it though if it interests you: >>> https://james.staged.apache.org/james-project/3.8.0/index.html >>> >>> Distributed doc in the source code though if you feel like building >>> it yourself: >>> https://github.com/apache/james-project/tree/master/server/apps/distributed-app/docs >>> >>> I think yes forwarding your mails to both your main and backup >>> servers is simple and efficient :) >>> >>>> >>>> Have a good one. >>>> >>>> Matt >>>> >>>> On 09.01.24 08:44, Rene Cordier wrote: >>>>> Hello, >>>>> >>>>> Interesting read thanks. >>>>> >>>>> Few remarks though, but feel free to agree or not :) : >>>>> >>>>> - any reason to still run the spring version of James? That >>>>> version I believe is not always up-to-date with dependencies >>>>> (specially the sprint dependencies). Truth is it might get >>>>> deprecated and removed in later release if a consensus is reached, >>>>> as all active dev is done on the Guice versions now. You might >>>>> want to check that (JPA Guice here probably closest to the Spring >>>>> version) >>>>> >>>>> - note that there is active development at the moment (still WIP) >>>>> regarding a Postgresql reactive implementation with james (Guice). >>>>> I saw you seem to use MariaDB but might interest you for the >>>>> future maybe? >>>>> >>>>> - for sync between your main server and backup, maybe just >>>>> redirect the mails to both platforms, and let them treat the mails >>>>> separately? If not I guess db backup and sync between the two >>>>> could do it too. >>>>> >>>>> Good job on running latest James on the new OpenSuse though, >>>>> thanks for sharing! >>>>> >>>>> Rene. >>>>> >>>>> On 1/9/24 10:17, cryptearth wrote: >>>>>> Hello there everybody, >>>>>> >>>>>> happy new year 2024 to all of you. >>>>>> It's been quite some time since I was last active on this >>>>>> maillist. After >>>>>> upgrading my server to OpenSuSE 15.5 and James to 3.8 and after >>>>>> reading the >>>>>> recent topics on the web archive I decided to post a small round >>>>>> up how it >>>>>> went this way. >>>>>> >>>>>> Important word of warning and security: >>>>>> This is only a personal write-down how I did my setup. I'm aware >>>>>> there're >>>>>> still some security improvements to be done. This is meant as an >>>>>> informative >>>>>> guide. I'm by no means responsible for any security issues that >>>>>> may arise by >>>>>> following this. I myself got hacked by the Log4J issue mostly due >>>>>> to not >>>>>> applying the available update fast enough. It's your own >>>>>> responsibility to >>>>>> make your setup secure. All I can offer are some guidelines from >>>>>> personal >>>>>> experience. >>>>>> >>>>>> That warning out now let's get to the real topic. >>>>>> >>>>>> As several of the active devs here know I use James for quite >>>>>> some time now >>>>>> and over the years encountered several issues mostly specific to >>>>>> OpenSuSE and >>>>>> the packages provided by its repositories. Luckily this time >>>>>> everything went >>>>>> smoothly. >>>>>> >>>>>> I started from a fresh clean "server"-type install, but this >>>>>> should be >>>>>> applicable to any existing system as well. So the first step is >>>>>> to fully update >>>>>> the new system. >>>>>> >>>>>> As OpenSuSE comes with postfix I replace it with sendmail for >>>>>> later. I guess >>>>>> it's also possible with postfix or exim or any other MTA >>>>>> providing the >>>>>> required sendmail dependency. Using sendmail was just the >>>>>> simplest option I >>>>>> figured out when I first used it. >>>>>> The keyword here is "nullclient": It configures sendmail to just >>>>>> forward >>>>>> everything to James without any handling itself. For this >>>>>> sendmail comes with >>>>>> an almost ready-to-use config which only needs one change in >>>>>> /etc/sendmail/ >>>>>> linux.nullclient.mc in line 27 by replacing the example >>>>>> "mailhub.domain.notused" with "localhost". Then just generate the >>>>>> new >>>>>> sendmail.cf from it (either as root or via sudo): >>>>>> m4 /etc/mail/linux.nullclient.mc > /etc/sendmail.cf >>>>>> >>>>>> I also have to edit the systemd script. For some reason the >>>>>> systemctl edit >>>>>> command works a bit unexpected for simple edits so by appending >>>>>> --full it >>>>>> becomes a full override. The service file needs two changes: >>>>>> >>>>>> 1) moving the PIDFile from /var/run to /run as hinted by >>>>>> systemctl status >>>>>> message: >>>>>> PIDFile=/run/sendmail.pid >>>>>> >>>>>> 2) remove the -bd flag from the start args to prevent sendmail >>>>>> from blocking >>>>>> port TCP/25: >>>>>> Environment="SENDMAIL_ARGS=-L -Am -q30m -om" >>>>>> >>>>>> Check if sendmail service is enabled and started: systemctl >>>>>> status sendmail >>>>>> At least enabled it so it gets started at reboot. As James isn't >>>>>> setup yet to >>>>>> take its mails don't start it yet - it could cause a hang into an >>>>>> infinite >>>>>> loop. >>>>>> >>>>>> Before get to James I prepare the rest of the system by install >>>>>> these packages >>>>>> (can be installed along when switching postfix for sendmail): >>>>>> - java-17-openjdk-devel >>>>>> - git >>>>>> - maven >>>>>> - mariadb >>>>>> - phpMyAdmin-apache >>>>>> - yast2-http-server >>>>>> The required dependencies get pulled in by the resolver. After >>>>>> installing the >>>>>> packages YaST tells you to secure MariaDB. So I do this right >>>>>> away by running >>>>>> mariadb-secure-installation as root/sudo and set a root password. >>>>>> All other >>>>>> questions can be answered with thier default option (just hit >>>>>> enter). >>>>>> >>>>>> To get phpMyAdmin running only the initial setup using "yast >>>>>> http-server" is >>>>>> required. Select enable PHP in the 2nd step and complete the >>>>>> setup by just >>>>>> hitting F10 a couple times. If the Apache2 server is accessible >>>>>> via the >>>>>> internet care should be taken by limiting phpMyAdmin to >>>>>> localhost. Edit /etc/ >>>>>> apache2/conf.d/phpMyAdmin.conf line 27 to "Require local" and use >>>>>> ssh tunnels. >>>>>> If you have services like a vpn or a tor hidden service running >>>>>> you may want >>>>>> to secure it even further with something like an .htaccess or >>>>>> similar. This >>>>>> way you prevent attackers from trying to bruteforce your database >>>>>> password. >>>>>> >>>>>> Complete the pre-setup by use phpMyAdmin to add a new user for >>>>>> James. >>>>>> >>>>>> Now let's build James: >>>>>> git clone https://github.com/apache/james-project -b >>>>>> james-project-3.8.0 >>>>>> cd james-project >>>>>> mvn clean package -DskipTests -T 1C >>>>>> As I use the spring package I then unpack the tar in >>>>>> james-project/server/ >>>>>> apps/spring-app/target/james-server-spring-app-3.8.0-app.tar.gz. >>>>>> I also need to get the mariadb-connector/J: >>>>>> https://mariadb.com/downloads/ >>>>>> connectors/ and place the .jar in the >>>>>> james-server-spring-app-3.8.0/lib >>>>>> folder. >>>>>> I'm not sure why this isn't distributed along or part of the >>>>>> package manifest >>>>>> to pull from the repos - guess this is some licence nonesense. >>>>>> >>>>>> It's config time now - and there's not that much to do. >>>>>> As I don't use POP3 the first I do is disable the POP3 server. >>>>>> Next I set up the database stuff by change the driver to >>>>>> org.mariadb.jdbc.Driver >>>>>> and the connector-string to >>>>>> jdbc:mariadb://localhost:3306/james >>>>>> Btw: As pretty much any modern JDBC driver should be a type 4.x >>>>>> there's no >>>>>> need to specify the driver class name or do something like >>>>>> Class.forName() >>>>>> As long as the driver jar is in classpath on startup of the JVM >>>>>> it gets pulled >>>>>> in via the ServiceLoader-API. >>>>>> >>>>>> Next I set my domain and the greetings in the imap and smtp >>>>>> servers. I also >>>>>> enable StartTLS for both imap and smtp. As for outgoing smtp I >>>>>> also enable >>>>>> StartTLS by adding this to the RemoteDelievery in the >>>>>> mailetcontainer.xml: >>>>>> <startTLS>true</startTLS> >>>>>> This should be added by default as it's only mentioned in the >>>>>> RemoteDelievery >>>>>> javadoc but not in the XML doc nor in the config doc. >>>>>> Until now I used a Java keystore for the certificate and key. But >>>>>> using the new >>>>>> PEM files directly skips this. Thanks for who added this. >>>>>> >>>>>> The last part is to deal with sendmail and apache2/php mail. For >>>>>> this I first >>>>>> add two users to James (along with others like postmaster): >>>>>> webmaster and >>>>>> local-service. Then I add this to the mailetcontainer.xml right >>>>>> at the top of >>>>>> the root processor: >>>>>> >>>>>> <matcher name="local-service" >>>>>> match="org.apache.james.mailetcontainer.impl.matchers.And"> >>>>>> <matcher match="RemoteAddrInNetwork=127.0.0.0/8" /> >>>>>> <matcher >>>>>> match="org.apache.james.mailetcontainer.impl.matchers.Not"> >>>>>> <matcher match="SMTPAuthSuccessful" /> >>>>>> </matcher> >>>>>> <matcher >>>>>> match="org.apache.james.mailetcontainer.impl.matchers.Not"> >>>>>> <matcher >>>>>> match="[email protected]" /> >>>>>> </matcher> >>>>>> <matcher >>>>>> match="org.apache.james.mailetcontainer.impl.matchers.Not"> >>>>>> <matcher match="RecipientIs=local- >>>>>> [email protected]" /> >>>>>> </matcher> >>>>>> </matcher> >>>>>> >>>>>> <mailet match="local-service" class="Redirect"> >>>>>> <to>[email protected]</to> >>>>>> </mailet> >>>>>> >>>>>> To explain its purpose: >>>>>> When starting James via crontab by root crond sends a start >>>>>> report via >>>>>> sendmail. In order for sendmail to not get stuck in an endless >>>>>> loop trying to >>>>>> deliver the crond status I have to provide some drop off >>>>>> destination for it. >>>>>> But as I also want to send e-mails by PHP I had to get a bit >>>>>> creative >>>>>> filtering. It basically goes like this: >>>>>> >>>>>> IF incoming mail comes from localhost >>>>>> AND is NOT authenticated via smtp auth >>>>>> AND is NOT sent by webmaster (from php) >>>>>> AND is NOT already redirected to local-service >>>>>> THEN redirect to local-service >>>>>> >>>>>> This rule maybe can be written better - but it was the simplest I >>>>>> came up >>>>>> with. The check for the recipient is required so the mail doesn't >>>>>> get >>>>>> redirected to itself again. The check for the webmaster is >>>>>> required as >>>>>> sendmail drops off php mails with wwwrun@localhost, which is >>>>>> overriden by the - >>>>>> f parameter to webmaster. Any other client that regular logs in >>>>>> via AUTH is >>>>>> ignored anyway. And the whole thing is limited to localhost. >>>>>> >>>>>> Finally, add a >>>>>> @reboot /path/to/james/bin/james start >>>>>> to roots crontab for james autostart. >>>>>> >>>>>> If everything was done correctly james should start up just fine >>>>>> for the first >>>>>> time (it actually already had to for adding the users - but this >>>>>> can be done >>>>>> now). >>>>>> I do the first start via "console" to verify everything is done >>>>>> correctly. >>>>>> After all is done and james starts correctly I just reboot the >>>>>> system. Now >>>>>> when all comes up by itself the local-service user should have a >>>>>> mail confirm >>>>>> the successful start of James which also confirms that the >>>>>> sendmail nullclient >>>>>> and the mailet redirect works. >>>>>> >>>>>> A common failure can be an issue with name resolution. In this >>>>>> case you have >>>>>> to manually set /etc/hosts and reboot again. This can also be >>>>>> done early on as >>>>>> one of the first few steps after updating the system. >>>>>> >>>>>> To send mails from apache this can be done like this: >>>>>> >>>>>> <?php >>>>>> $to="[email protected]"; >>>>>> $subject=$content="this is a apache webmail test"; >>>>>> $headers = array( >>>>>> 'From' => 'CryptEarth.de Webmaster >>>>>> <[email protected]>', >>>>>> 'Reply-To' => '[email protected]', >>>>>> 'X-Mailer' => 'PHP/' . phpversion() >>>>>> ); >>>>>> mail($to, $subject, $content, $headers, >>>>>> "[email protected]"); >>>>>> ?> >>>>>> >>>>>> A few improvements that can (and should) be done: >>>>>> >>>>>> - don't start james as root but rather move the ports to 8025 and >>>>>> 8143 and use >>>>>> a firewall rule to redirect 25 to 8025 and 143 to 8143 ... but I >>>>>> have to figure >>>>>> that out >>>>>> - maybe rethink about sendmail nullclient and do similar with >>>>>> postfix or exim >>>>>> - maybe redo the mailet redirect for local-service and php-mail >>>>>> - setting up DKIM on outgoing mails >>>>>> >>>>>> Currently I have James running on two servers: my main root as >>>>>> primary and on >>>>>> a small sbc at home as a backup. Is there a way to synchronize >>>>>> the two servers >>>>>> on the application level. Or would it be enough to just >>>>>> synchronize the two >>>>>> datases secured by a ssh tunnel? How to handle the differences in >>>>>> the counters >>>>>> when both servers receive an e-mail each which have to get >>>>>> synchronized in >>>>>> both ways? >>>>>> >>>>>> Have a nice week. >>>>>> Greetings from Germany >>>>>> >>>>>> Matt >>>>>> >>>>>> >>>>>> >>>>>> --------------------------------------------------------------------- >>>>>> >>>>>> To unsubscribe, e-mail: [email protected] >>>>>> For additional commands, e-mail: [email protected] >>>>>> >>>>>> >>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: [email protected] >>>>> For additional commands, e-mail: [email protected] >>>>> >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: [email protected] >>>> For additional commands, e-mail: [email protected] >>>> >>>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [email protected] >>> For additional commands, e-mail: [email protected] >>> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] >