Re: 3.9.0 and S3 BlobStore

--000000000000eb9539064762c9da
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi jerry,

Thanks for providing yet another example of how useless genai can be.

There is a sample configuration included in the zip file :
database.name=3Djames
database.schema=3Dpublic
database.host=3Dpostgres
database.port=3D5432
database.username=3Djames
database.password=3Dsecret1


cheers
jean

On Fri, Jan 2, 2026 at 1:49=E2=80=AFAM Jerry Malcolm <[email protected]=
m> wrote:

> Thanks again.  You convinced me.  PostGreSQL it is.  I've spent the day
> digging deep into postgres and getting it installed, with the
> postgres/guice package 'almost' up and running.
>
> But (as usual) I hit one snag that I can't seem to get around. I've
> created postgres.properties and blob.properties in /opt/james/var/conf
> and set var/conf as the working directory.  I know it's processing the
> blob.properties (i had an omission, and it complained until i fixed
> it).  It screamed loudly when it initially couldn't find
> postgres.properties, and those errors went away when I created the
> postgres.properties file.  So it is definitely now finding that file.
> However, no matter what I put in that file (including one time with just
> trash characters), I always get the same error:
>
> 00:24:45.757 [main] INFO  org.apache.james.CONFIGURATION - Load
> configuration file /opt/james/var/conf/postgres.properties
> Exception in thread "main" java.lang.IllegalArgumentException: You need
> to specify username
>
> ChatGPT has told me two different formats for that file.  But neither
> makes any difference.  Always the same error above.  What am I doing wron=
g?
>
> version 1:
>
> host=3Ddb_pg_jwmbos.******.com
> port=3D5432
> database=3Djamesdb
> username=3D********
> password=3D***********
> maxConnections=3D10
>
> version 2:
>
> postgres.host=3Ddb_pg_jwmbos.******.com
> postgres.port=3D5432
> postgres.database=3Djamesdb
> postgres.username=3D*********
> postgres.password=3D*********
> postgres.maxConnections=3D10
>
>
>
>
> On 1/1/2026 11:23 AM, Jean Helou wrote:
> > Hi Jerry,
> >
> >> I'm doing a completely clean install, and I'm already set up to use
> > imapsync to populate the new instance.  So no  worries about migration.
> >
> > good to know
> >
> >> don't believe I care about spring vs guice.  I've never used a guice
> > version.  But spring-->guice should be transparent unless I've done
> custom
> > spring stuff, correct (which I haven't)?
> >
> > Exactly, unless you have custom spring extensions for james using eithe=
r
> is
> > transparent for you
> >
> >> You said there were only the 4 pre-packaged assemblies with S3.  Does
> > that mean that (with a little work on my part), there 'could' be a
> > custom-packaged assembly created that would support mysql/s3?
> >
> > In theory: yes. However, I looked into how big a work it would be this
> > morning while writing my answer to see if I should suggest it and
> > unfortunately I expect it to be quite involved.
> >
> > there are 2 aspects :
> > - "short" term storage of mailqueue messages (mails that are being
> received
> > and processed or mails that are enqueued for sending, plus the various
> > error queues which may not be that short :D )
> > - long term storage of mailbox messages
> >
> > The first aspect can be managed through (a bit of tortuous) configurati=
on
> > but the second requires code modifications
> > The only mysql support is based on openJPA [1] which is a generic java
> > persistence system. The code that writes the message to the database wh=
en
> > using openJPA is org.apache.james.mailbox.jpa.mail.JPAMessageMapper if
> you
> > look at it you will see that the implementation is quite straightforwar=
d
> in
> > the protected `org.apache.james.mailbox.jpa.mail.JPAMessageMapper#save`
> > method
> > The org.apache.james.mailbox.postgres.mail.PostgresMessageMapper is
> > structured a bit differently and if you look at the save method, it fir=
st
> > stores the message body ( private method saveBodyContent) then asks
> > the org.apache.james.mailbox.postgres.mail.dao.PostgresMessageDAO#inser=
t
> to
> > store the message providing it with the message from which it extracts
> the
> > headers and the blobid of the saved body content.
> >
> > To enable such behaviour you would need to create a custom
> JPAMessageMapper
> > and probably a custom variant of JPAMailboxMessage that has a field to
> > store the blobId, create the correspond SQL migration to add the missin=
g
> > column and do the dance around saving the body, then saving the message
> > etc. Since the MessageMapper is not created directly by the IOC contain=
er
> > you will need to implement a custom JPAMailboxSessionMapperFactory
> > Once you have done this, you will need to make the IOC container pick
> your
> > factory implementation over the standard one (whether using spring or
> > guice)
> >
> > An alternative but probably requiring even more work would be to create=
 a
> > Mysql connector using Jooq (same as the postgres connector) thats
> probably
> > a lot more code.
> >
> > best regards,
> > Jean
> >
> >
> >
> >
> > On Thu, Jan 1, 2026 at 4:50=E2=80=AFPM Jerry Malcolm <techstuff@malcolm=
s.com>
> wrote:
> >
> >> Jean,
> >>
> >> Thank you so much for the detailed explanation.  One clarification tha=
t
> >> might make decisions easier... I'm doing a completely clean install, a=
nd
> >> I'm already set up to use imapsync to populate the new instance.  So n=
o
> >> worries about migration.
> >>
> >> I don't believe I care about spring vs guice.  I've never used a guice
> >> version.  But spring-->guice should be transparent unless I've done
> >> custom spring stuff, correct (which I haven't)?
> >>
> >> So it really comes down to mysql vs. postgres, correct?  I've never us=
ed
> >> postgres.  But installing and learning it is doable. But before I head
> >> down that path, I would like to understand any other possibilities.  Y=
ou
> >> said there were only the 4 pre-packaged assemblies with S3.  Does that
> >> mean that (with a little work on my part), there 'could' be a
> >> custom-packaged assembly created that would support mysql/s3?    I'm a
> >> programmer, but I don't purport to know the inner workings of James
> >> wiring.  But just for exploration purposes, how big a project would it
> >> be to make james use mysql with s3 instead of postgres with s3
> >> (obviously requiring some code changes and wiring).  But I'm just
> >> curious if this is something that is doable on my part?  If it's massi=
ve
> >> changes... then on to postgres.  But I'm not adverse to a little codin=
g,
> >> especially if it reduces the learning curve for a new db environment.
> >>    Or if there are plans anytime in the immediate future for james to
> >> release a mysql/s3 option, then I might just pay a bit more for db
> >> storage and wait it out.
> >>
> >> Again, I appreciate you taking time on New Years Day to answer my
> >> questions  Thanks again.
> >>
> >> Jerry
> >>
> >>
> >> On 1/1/2026 3:38 AM, Jean Helou wrote:
> >>> Hello jerry,
> >>>
> >>> I'm afraid you are a bit out of luck: james has a modular architectur=
e
> >> and
> >>> there are multiple implementations for most if not all modules, this
> >> means
> >>> that the number of module combination is very high.
> >>> the packaged assemblies cover some of the combinations, some assembli=
es
> >> are
> >>> configurable to make it possible to cover even more through
> >> configuration (
> >>> at the cost of increased package size )
> >>>
> >>> Unfortunately there is no pre-packaged assembly which supports both
> MySQL
> >>> AND S3 blobstore today.
> >>>
> >>> Only 4 assemblies have blobstore capabilities :
> >>> - james distributed
> >>> - james distributed pop3
> >>> - james postgresql
> >>> - scaling pulsar smtp ( this is a demo assembly for an SMTP only serv=
er
> >>> which uses the pulsar message broker/postgres and s3 )
> >>> so that really only leaves 2 options:
> >>> - james distributed
> >>> - james postgresql
> >>>
> >>> The former is designed for large enterprise deployments and relies on
> >>> complex systems
> >>> The latter is designed for small to medium deployments
> >>>
> >>> Given you are using the spring/mysql james server  today and only wan=
t
> s3
> >>> to optimize storage costs, I think your best bet is to go with  james
> >>> postgresql. The distributed version is designed for enterprise with
> tens
> >> or
> >>> hundreds of thousands of users, or for high volume servers. Cassandra
> is
> >>> not easy to operate.
> >>>
> >>> Unless you customized it quite heavily, the spring assembly should be
> >> using
> >>> the jpa implementation to access the database, this means you can use
> the
> >>> provided migration tool[1]  to move you data from JPA to Postgres.
> >>> You will have to build the migration tool from source as it is not
> >> included
> >>> in the officially published artifacts for the project.
> >>> This is a one shot operation after which you will be able to use the
> >>> james-server-postgres-app assembly which supports both postgres and s=
3
> >>> blobstore.
> >>>
> >>> The postgres server is an officially published artifact[2]  (there is=
 a
> >> bug
> >>> in the website as the title for that link is
> >>> also  james-server-distributed-guice.zip but its only the title which
> is
> >>> incorrect, the link goes to the right place)
> >>> You can run both servers side by side for a while, repeatedly running
> the
> >>> migration tool (it is designed for this) until you are satisfied the
> >>> migration is complete with the limitation that the migration only goe=
s
> >> one
> >>> way so mails you send/receive  with the postgres server will never be
> >> known
> >>> by the mysql server.
> >>> A basic configuration will
> >>> - store queued messages (either incoming or outgoing) in postgres
> >>> - store mailbox messages (long term storage for received or sent
> >> messages)
> >>> in s3
> >>> further configuration can store queued messages in s3 too if you want
> >>>
> >>> With regards to your compilation problems it's hard to tell without
> >> seeing
> >>> the actual errors, a full compilation running all tests takes about 5
> >> hours
> >>> (there are a lot of rather slow integration tests when you build
> >>> everything), so you probably want to either skip the tests or target =
a
> >>> specific project
> >>>
> >>> mvn clean package -DskipTests will build everything without running t=
he
> >>> tests
> >>> mvn clean package -am -pl :$project-name will build a project and all
> its
> >>> dependencies running all the tests for all the built projects, you ca=
n
> >> also
> >>> add the -DskipTests to avoid running the tests
> >>>
> >>> if you encounter further errors don't hesitate to upload the output t=
o
> a
> >>> pastebin or filebin of some kind and include the link, the last build
> for
> >>> master[3] and the 3.9.x[4] branch on the james CI are green so the
> build
> >>> should be ok
> >>>
> >>> Cheers,
> >>> Jean
> >>>
> >>> 1
> >>>
> >>
> https://github.com/apache/james-project/tree/master/server/apps/migration=
/core-data-jpa-to-pg
> >>> 2
> >>>
> >>
> https://www.apache.org/dyn/closer.lua/james/server/3.9.0/james-server-pos=
tgres-app.zip
> >>> 3 https://ci-builds.apache.org/job/james/job/ApacheJames/job/master/
> >>> 4 https://ci-builds.apache.org/job/james/job/ApacheJames/job/3.9.x/
> >>>
> >>> On Wed, Dec 31, 2025 at 7:31=E2=80=AFPM Jerry Malcolm <techstuff@malc=
olms.com>
> >>> wrote:
> >>>
> >>>> I am so confused.  I have been trying for 5 days to upgrade to James
> >>>> 3.9.0 and begin using AWS S3 blobstore.   I've been using ChatGPT fo=
r
> >>>> guidance.  But I'm beginning to realize that ChatGPT is just making =
up
> >>>> answers on the fly, giving me completely different and wrong answers
> >>>> depending on the time of day.
> >>>>
> >>>> - first answer from ChatGPT - I should install 3.9.0 spring with
> >>>> JPA/mySQL and configure AWS S3 for blob storage (never could get jam=
es
> >>>> to even recognize that I had blob storage configured)
> >>>>
> >>>>     - revised answer from ChatGPT - Oh, you can't use blobstore with
> >>>> JPA/mySQL... you have to get Cassandra version to use blobstore....
> took
> >>>> over a day to make cassandra happy with it's Java 17 requirement whi=
le
> >>>> keeping James 3.9.0 happy with its required Java 21.   Now I can't
> find
> >>>> the 'required' James 3.9.0 Spring-Cassandra version zipfile to
> download
> >>>>
> >>>> - revised answer from ChatGPT - Oh, there is no Spring/Cassandra....
> you
> >>>> need to use 3.9.0 Guice/Cassandra, which is not recommended since
> Spring
> >>>> is strategic even though with last release spring was going away....=
.
> >>>> <head exploding icon>
> >>>>
> >>>> I'm sick of 'recommendation du-jour' from ChatGPT...  I've been on
> James
> >>>> / James-spring for over 20 years with mySQL.  I simply want to cut m=
y
> >>>> horrendous RDS/mySQL expenses for James mail data by using S3 for bl=
ob
> >>>> storage.  I'll use any configuration available that I can get to wor=
k
> as
> >>>> long as I can have S3 blob storage.
> >>>>
> >>>> Oh, yeah... and I cloned the 3.9.0 release source from GitHub and
> tried
> >>>> to run maven.  maven ran for about 20 minutes, then crashed and burn=
ed
> >>>> on a bunch of errors.  So I had to go with the pre-built 3.9.0 zip.
> >>>>
> >>>> I have just one simple question.... What are my options for
> >>>> configuration in order to use AWS S3 for blob storage.
> >>>>
> >>>> Thx
> >>>>
> >>>> Jerry
> >>>>
> >>>>
> >>>> --------------------------------------------------------------------=
-
> >>>> To unsubscribe, e-mail: [email protected]
> >>>> For additional commands, e-mail: [email protected]
> >>>>
> >>>>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: [email protected]
> >> For additional commands, e-mail: [email protected]
> >>
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
>
>

--000000000000eb9539064762c9da--