Cyrus replica 2.4 to 3.2.2 issues

Hello, I'm trying to replicate from Cyrus IMAP 2.4.20 to Cyrus IMAP 
3.2.2 using replication.

Because Cyrus 2.4 doesn't support IMAP replication (is this true?), I 
configured the replica with

replica2to3syncserver cmd="/usr/libexec/cyrus-imapd/sync_server" 
proto=tcp4 listen="csync"

in SERVICES of cyrus.conf.

The sync_client in master works in rolling mode, such as

/usr/lib/cyrus-imapd/sync_client -r -n replica2-3

Both master and replica hosts work with delayed deleted and delayed 
expunge fashion.

I experience some problems.
In the below logs tst-msg01 is the master, tst-msg03 is the replica.

== 1 ==
I found "errors" of this kind:

2020-07-21T15:05:16.803765+02:00 tst-msg03 
cyrus/replica2to3syncserver[1238 sync_log(): Failed to lock 
/var/lib/imap/sync/squatter/log for MAILBOX 
example.com!user.demo^gruppoarchivio.ticket.Vecchi#012 after 65 attempts
2020-07-21T15:05:16.803898+02:00 tst-msg03 
cyrus/replica2to3syncserver[1238 mailbox: longlock 
example.com!user.demo^gruppoarchivio.ticket.Vecchi for 2.5 seconds
2020-07-21T15:05:17.090124+02:00 tst-msg03 
cyrus/replica2to3syncserver[1238 sync_log(): Failed to lock 
/var/lib/imap/sync/squatter/log for MAILBOX 
example.com!user.demo^gruppoarchivio.ticket.Vecchi#012 after 65 attempts
2020-07-21T15:05:17.101288+02:00 tst-msg03 
cyrus/replica2to3syncserver[1238 mappedfile: longlock 
/var/lib/imap/domain/Q/example.com/user/E/demo.gruppoarchivio.conversations 
for 2.7 seconds
2020-07-21T15:05:17.101410+02:00 tst-msg03 cyrus/squatter[27276]: 
indexing mailbox user/demo.gruppoarchivio/ticket/[email protected]...
2020-07-21T15:05:18.132371+02:00 tst-msg03 cyrus/squatter[27276]: Xapian 
committed 740 updates in 0.374236 sec
2020-07-21T15:05:18.145507+02:00 tst-msg03 cyrus/squatter[27276]: 
mappedfile: longlock 
/var/lib/imap/domain/Q/example.com/user/E/demo.gruppoarchivio.xapianactive 
for 1.0 seconds

But the xapian index seems to be ok:

-rw------- 1 cyrus mail      864 Jul 21 15:05 cyrus.indexed.db
-rw------- 1 cyrus mail        0 Jul 21 15:05 flintlock
-rw------- 1 cyrus mail      116 Jul 21 15:05 iamglass
-rw------- 1 cyrus mail 13115392 Jul 21 15:05 position.glass
-rw------- 1 cyrus mail  5545984 Jul 21 15:05 postlist.glass
-rw------- 1 cyrus mail  3530752 Jul 21 15:05 termlist.glass

I hope I can ignore this. I've found these errors only for one mailbox.


== 2 ==
Another problem is that the "expire" annotation value is not configured 
as an "expire" metadata value on Cyrus 3.2.2. I don't know if this is 
expected or if this is a bug. But it is a problem for me, I have to 
rewrite all these mailboxes metadata.

To be more clear, the Cyrus IMAP 2.4.20 annotation like

* ANNOTATION "user/marco/[email protected]" 
"/vendor/cmu/cyrus-imapd/expire" ("value.shared" "14" 
"content-type.shared" "text/plain" "size.shared" "2" 
"modifiedsince.shared" "1406624279")

become in Cyrus IMAP 3.2.2 after the sync:
* METADATA user/marco/[email protected] 
("/shared/vendor/cmu/cyrus-imapd/expire" NIL)

I expected "14" and not NIL.

== 3 ==
The specialuse flags are not kept in the replica server:

* LIST (\HasNoChildren \Sent) "/" INBOX/Sent
become
* LIST (\HasNoChildren) "/" INBOX/Sent

Do I have to suppose that I must run the cvt_xlist_specialuse tool as 
described in 
https://www.cyrusimap.org/3.2/imap/download/upgrade.html#upgrade-specific-items 
?
It's not clear to me if this instruction is related to the in place 
upgrade only or to the replication mode too.


== 4 ==
This is a severe error. When I expunge a message in the master, the 
expunge action is not replicated on the replica, and the sync_client dies:

2020-07-23T14:50:44.695378+02:00 tst-msg01 cyrus/imap[26851]: Expunged 1 
messages from example.com!user.marco^fff.Trash
2020-07-23T14:51:10.147600+02:00 tst-msg03 
cyrus/replica2to3syncserver[1207 Fatal error: Internal error: assertion 
failed: imap/dlist.c: 156: base != NULL
2020-07-23T14:51:10.693261+02:00 tst-msg01 cyrus/sync_client[26834]: 
RESERVE received * response:
2020-07-23T14:51:10.693321+02:00 tst-msg01 cyrus/sync_client[26834]: 
reserve messages: failed: Bad protocol
2020-07-23T14:51:10.693730+02:00 tst-msg01 cyrus/sync_client[26834]: 
Error in do_sync(): bailing out! Bad protocol
2020-07-23T14:51:10.693750+02:00 tst-msg01 cyrus/sync_client[26834]: 
Processing sync log file /var/lib/imap/sync/replica2-3/log-26834 failed: 
Bad protocol
2020-07-23T14:51:10.706750+02:00 tst-msg03 cyrus/master[24009]: process 
type:SERVICE name:replica2to3syncserver 
path:/usr/libexec/cyrus-imapd/sync_server age:1142.993s pid:1207 exited, 
status 70

I can't no more suppress this error. All sync_client die, even in non 
rolling mode. I have to delete permanently the replicated mailboxes to 
resume the right working.

I see the same error when I delete a mailbox on replica and after that I 
do a sync_client on master. In this last case the error disappear only 
when I delete the delayed DELETED mailbox too.

I read the documentation, I didn't find restrictions on replica with the 
delayed expunged or the delayed deleted configuration in master and or 
in replica.
In my environment I simply configured delayed deleted and delayed 
expunge in the same way, on master and on replica hosts.


Could you help me to understand?
Thank you very much

Cheers
Marco
----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus