Re: 3.9.0 and S3 BlobStore

Thanks again.=C2=A0 You convinced me.=C2=A0 PostGreSQL it is.=C2=A0 I've sp=
ent the day=20
digging deep into postgres and getting it installed, with the=20
postgres/guice package 'almost' up and running.

But (as usual) I hit one snag that I can't seem to get around. I've=20
created postgres.properties and blob.properties in /opt/james/var/conf=20
and set var/conf as the working directory.=C2=A0 I know it's processing the=
=20
blob.properties (i had an omission, and it complained until i fixed=20
it).=C2=A0 It screamed loudly when it initially couldn't find=20
postgres.properties, and those errors went away when I created the=20
postgres.properties file.=C2=A0 So it is definitely now finding that file.=
=C2=A0=20
However, no matter what I put in that file (including one time with just=20
trash characters), I always get the same error:

00:24:45.757 [main] INFO=C2=A0 org.apache.james.CONFIGURATION - Load=20
configuration file /opt/james/var/conf/postgres.properties
Exception in thread "main" java.lang.IllegalArgumentException: You need=20
to specify username

ChatGPT has told me two different formats for that file.=C2=A0 But neither=
=20
makes any difference.=C2=A0 Always the same error above.=C2=A0 What am I do=
ing wrong?

version 1:

host=3Ddb_pg_jwmbos.******.com
port=3D5432
database=3Djamesdb
username=3D********
password=3D***********
maxConnections=3D10

version 2:

postgres.host=3Ddb_pg_jwmbos.******.com
postgres.port=3D5432
postgres.database=3Djamesdb
postgres.username=3D*********
postgres.password=3D*********
postgres.maxConnections=3D10




On 1/1/2026 11:23 AM, Jean Helou wrote:
> Hi Jerry,
>
>> I'm doing a completely clean install, and I'm already set up to use
> imapsync to populate the new instance.  So no  worries about migration.
>
> good to know
>
>> don't believe I care about spring vs guice.  I've never used a guice
> version.  But spring-->guice should be transparent unless I've done  cust=
om
> spring stuff, correct (which I haven't)?
>
> Exactly, unless you have custom spring extensions for james using either =
is
> transparent for you
>
>> You said there were only the 4 pre-packaged assemblies with S3.  Does
> that mean that (with a little work on my part), there 'could' be a
> custom-packaged assembly created that would support mysql/s3?
>
> In theory: yes. However, I looked into how big a work it would be this
> morning while writing my answer to see if I should suggest it and
> unfortunately I expect it to be quite involved.
>
> there are 2 aspects :
> - "short" term storage of mailqueue messages (mails that are being receiv=
ed
> and processed or mails that are enqueued for sending, plus the various
> error queues which may not be that short :D )
> - long term storage of mailbox messages
>
> The first aspect can be managed through (a bit of tortuous) configuration
> but the second requires code modifications
> The only mysql support is based on openJPA [1] which is a generic java
> persistence system. The code that writes the message to the database when
> using openJPA is org.apache.james.mailbox.jpa.mail.JPAMessageMapper if yo=
u
> look at it you will see that the implementation is quite straightforward =
in
> the protected `org.apache.james.mailbox.jpa.mail.JPAMessageMapper#save`
> method
> The org.apache.james.mailbox.postgres.mail.PostgresMessageMapper is
> structured a bit differently and if you look at the save method, it first
> stores the message body ( private method saveBodyContent) then asks
> the org.apache.james.mailbox.postgres.mail.dao.PostgresMessageDAO#insert =
to
> store the message providing it with the message from which it extracts th=
e
> headers and the blobid of the saved body content.
>
> To enable such behaviour you would need to create a custom JPAMessageMapp=
er
> and probably a custom variant of JPAMailboxMessage that has a field to
> store the blobId, create the correspond SQL migration to add the missing
> column and do the dance around saving the body, then saving the message
> etc. Since the MessageMapper is not created directly by the IOC container
> you will need to implement a custom JPAMailboxSessionMapperFactory
> Once you have done this, you will need to make the IOC container pick you=
r
> factory implementation over the standard one (whether using spring or
> guice)
>
> An alternative but probably requiring even more work would be to create a
> Mysql connector using Jooq (same as the postgres connector) thats probabl=
y
> a lot more code.
>
> best regards,
> Jean
>
>
>
>
> On Thu, Jan 1, 2026 at 4:50=E2=80=AFPM Jerry Malcolm <techstuff@malcolms.=
com> wrote:
>
>> Jean,
>>
>> Thank you so much for the detailed explanation.  One clarification that
>> might make decisions easier... I'm doing a completely clean install, and
>> I'm already set up to use imapsync to populate the new instance.  So no
>> worries about migration.
>>
>> I don't believe I care about spring vs guice.  I've never used a guice
>> version.  But spring-->guice should be transparent unless I've done
>> custom spring stuff, correct (which I haven't)?
>>
>> So it really comes down to mysql vs. postgres, correct?  I've never used
>> postgres.  But installing and learning it is doable. But before I head
>> down that path, I would like to understand any other possibilities.  You
>> said there were only the 4 pre-packaged assemblies with S3.  Does that
>> mean that (with a little work on my part), there 'could' be a
>> custom-packaged assembly created that would support mysql/s3?    I'm a
>> programmer, but I don't purport to know the inner workings of James
>> wiring.  But just for exploration purposes, how big a project would it
>> be to make james use mysql with s3 instead of postgres with s3
>> (obviously requiring some code changes and wiring).  But I'm just
>> curious if this is something that is doable on my part?  If it's massive
>> changes... then on to postgres.  But I'm not adverse to a little coding,
>> especially if it reduces the learning curve for a new db environment.
>>    Or if there are plans anytime in the immediate future for james to
>> release a mysql/s3 option, then I might just pay a bit more for db
>> storage and wait it out.
>>
>> Again, I appreciate you taking time on New Years Day to answer my
>> questions  Thanks again.
>>
>> Jerry
>>
>>
>> On 1/1/2026 3:38 AM, Jean Helou wrote:
>>> Hello jerry,
>>>
>>> I'm afraid you are a bit out of luck: james has a modular architecture
>> and
>>> there are multiple implementations for most if not all modules, this
>> means
>>> that the number of module combination is very high.
>>> the packaged assemblies cover some of the combinations, some assemblies
>> are
>>> configurable to make it possible to cover even more through
>> configuration (
>>> at the cost of increased package size )
>>>
>>> Unfortunately there is no pre-packaged assembly which supports both MyS=
QL
>>> AND S3 blobstore today.
>>>
>>> Only 4 assemblies have blobstore capabilities :
>>> - james distributed
>>> - james distributed pop3
>>> - james postgresql
>>> - scaling pulsar smtp ( this is a demo assembly for an SMTP only server
>>> which uses the pulsar message broker/postgres and s3 )
>>> so that really only leaves 2 options:
>>> - james distributed
>>> - james postgresql
>>>
>>> The former is designed for large enterprise deployments and relies on
>>> complex systems
>>> The latter is designed for small to medium deployments
>>>
>>> Given you are using the spring/mysql james server  today and only want =
s3
>>> to optimize storage costs, I think your best bet is to go with  james
>>> postgresql. The distributed version is designed for enterprise with ten=
s
>> or
>>> hundreds of thousands of users, or for high volume servers. Cassandra i=
s
>>> not easy to operate.
>>>
>>> Unless you customized it quite heavily, the spring assembly should be
>> using
>>> the jpa implementation to access the database, this means you can use t=
he
>>> provided migration tool[1]  to move you data from JPA to Postgres.
>>> You will have to build the migration tool from source as it is not
>> included
>>> in the officially published artifacts for the project.
>>> This is a one shot operation after which you will be able to use the
>>> james-server-postgres-app assembly which supports both postgres and s3
>>> blobstore.
>>>
>>> The postgres server is an officially published artifact[2]  (there is a
>> bug
>>> in the website as the title for that link is
>>> also  james-server-distributed-guice.zip but its only the title which i=
s
>>> incorrect, the link goes to the right place)
>>> You can run both servers side by side for a while, repeatedly running t=
he
>>> migration tool (it is designed for this) until you are satisfied the
>>> migration is complete with the limitation that the migration only goes
>> one
>>> way so mails you send/receive  with the postgres server will never be
>> known
>>> by the mysql server.
>>> A basic configuration will
>>> - store queued messages (either incoming or outgoing) in postgres
>>> - store mailbox messages (long term storage for received or sent
>> messages)
>>> in s3
>>> further configuration can store queued messages in s3 too if you want
>>>
>>> With regards to your compilation problems it's hard to tell without
>> seeing
>>> the actual errors, a full compilation running all tests takes about 5
>> hours
>>> (there are a lot of rather slow integration tests when you build
>>> everything), so you probably want to either skip the tests or target a
>>> specific project
>>>
>>> mvn clean package -DskipTests will build everything without running the
>>> tests
>>> mvn clean package -am -pl :$project-name will build a project and all i=
ts
>>> dependencies running all the tests for all the built projects, you can
>> also
>>> add the -DskipTests to avoid running the tests
>>>
>>> if you encounter further errors don't hesitate to upload the output to =
a
>>> pastebin or filebin of some kind and include the link, the last build f=
or
>>> master[3] and the 3.9.x[4] branch on the james CI are green so the buil=
d
>>> should be ok
>>>
>>> Cheers,
>>> Jean
>>>
>>> 1
>>>
>> https://github.com/apache/james-project/tree/master/server/apps/migratio=
n/core-data-jpa-to-pg
>>> 2
>>>
>> https://www.apache.org/dyn/closer.lua/james/server/3.9.0/james-server-po=
stgres-app.zip
>>> 3 https://ci-builds.apache.org/job/james/job/ApacheJames/job/master/
>>> 4 https://ci-builds.apache.org/job/james/job/ApacheJames/job/3.9.x/
>>>
>>> On Wed, Dec 31, 2025 at 7:31=E2=80=AFPM Jerry Malcolm <techstuff@malcol=
ms.com>
>>> wrote:
>>>
>>>> I am so confused.  I have been trying for 5 days to upgrade to James
>>>> 3.9.0 and begin using AWS S3 blobstore.   I've been using ChatGPT for
>>>> guidance.  But I'm beginning to realize that ChatGPT is just making up
>>>> answers on the fly, giving me completely different and wrong answers
>>>> depending on the time of day.
>>>>
>>>> - first answer from ChatGPT - I should install 3.9.0 spring with
>>>> JPA/mySQL and configure AWS S3 for blob storage (never could get james
>>>> to even recognize that I had blob storage configured)
>>>>
>>>>     - revised answer from ChatGPT - Oh, you can't use blobstore with
>>>> JPA/mySQL... you have to get Cassandra version to use blobstore.... to=
ok
>>>> over a day to make cassandra happy with it's Java 17 requirement while
>>>> keeping James 3.9.0 happy with its required Java 21.   Now I can't fin=
d
>>>> the 'required' James 3.9.0 Spring-Cassandra version zipfile to downloa=
d
>>>>
>>>> - revised answer from ChatGPT - Oh, there is no Spring/Cassandra.... y=
ou
>>>> need to use 3.9.0 Guice/Cassandra, which is not recommended since Spri=
ng
>>>> is strategic even though with last release spring was going away.....
>>>> <head exploding icon>
>>>>
>>>> I'm sick of 'recommendation du-jour' from ChatGPT...  I've been on Jam=
es
>>>> / James-spring for over 20 years with mySQL.  I simply want to cut my
>>>> horrendous RDS/mySQL expenses for James mail data by using S3 for blob
>>>> storage.  I'll use any configuration available that I can get to work =
as
>>>> long as I can have S3 blob storage.
>>>>
>>>> Oh, yeah... and I cloned the 3.9.0 release source from GitHub and trie=
d
>>>> to run maven.  maven ran for about 20 minutes, then crashed and burned
>>>> on a bunch of errors.  So I had to go with the pre-built 3.9.0 zip.
>>>>
>>>> I have just one simple question.... What are my options for
>>>> configuration in order to use AWS S3 for blob storage.
>>>>
>>>> Thx
>>>>
>>>> Jerry
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: [email protected]
>>>> For additional commands, e-mail: [email protected]
>>>>
>>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [email protected]
>> For additional commands, e-mail: [email protected]
>>
>>