Re: Backup compaction optimization in a block-level replicatio n environment
"ellie timoney" <[email protected]>
| Newsgroups | gmane.mail.imap.cyrus |
|---|---|
| Message-ID | <[email protected]> |
On Tue, Nov 19, 2019, at 9:38 AM, Deborah Pickett wrote: > > Food for thought. Maybe instead of having one "%SHARED" backup, having one "%SHARED.foo" backup per top-level shared folder would be a better implementation? I haven't seen shared folders used much in practice, so it's interesting to hear about it. > > > > Looking at your own data, if you had one "%SHARED.foo" backup per top level shared folder, would they be roughly user-sized pieces, or still too big? If too big, how deep would you need to go down the tree until the worst offenders are a manageable size? (If I make it split shared folders like this, maybe "how-deep-to-split-shared-folders" needs to be a configuration parameter, because I guess it'll vary from installation to installation.) > > > For my data, %SHARED.foo would be the perfect granularity level. Each > foo is a shared email address like "sales" or "accounts" and it gets > about as much traffic as a user account does. (Two months ago when we > were on Exchange, they _were_ user accounts.) Ah yep, that makes sense! > foo also includes "#calendars" and "#addressbooks" on my server so there > are weird characters to deal with. Now that's an interesting detail to consider. I think, with a hypothetical depth setting, I would treat any level that contains '#directories' as being "too deep" for splitting, regardless of the depth setting, because at that point we're looking at things that I guess we expect to belong together. Like, if hypothetical_depth is 3, but foo.#calendars exists, then I think we'd want to treat the entirety of foo as a single backup (as if hypothetical_depth were 1), regardless of what else is deeper in there. I need to think about this more. I'm gonna have a go at implementing this (I've opened https://github.com/cyrusimap/cyrus-imapd/issues/2915) but I'll step through it one level of complication at a time. ---- Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus