Re: State of BDB FS backend
Ivan Zhakov <[email protected]> Wed, 29 Jul 2026 01:00:19 +0300
| Newsgroups | gmane.comp.version-control.subversion.devel |
|---|---|
| Message-ID | <CAPZho0-06RwJDX5E9taTA6VEFbgpJMZ1JHvNvpJWnBDDjWNpvQ@mail.gmail.com> |
--000000000000b569550657b2f79e Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Tue, 28 Jul 2026 at 18:22, Nathan Hartman <[email protected]> wrote: > On Tue, Jul 28, 2026 at 9:58=E2=80=AFAM Nathan Hartman <hartman.nathan@gm= ail.com> > wrote: > > > > On Tue, Jul 28, 2026 at 8:10=E2=80=AFAM Daniel Sahlberg > > <[email protected]> wrote: > > > > > > Den fre 24 juli 2026 kl 21:03 skrev Daniel Sahlberg < > [email protected]>: > > >> > > >> Den fre 24 juli 2026 kl 18:52 skrev Nathan Hartman < > [email protected]>: > > >>> > > >>> On Fri, Jul 24, 2026 at 11:52=E2=80=AFAM Ivan Zhakov <[email protected]= rg> > wrote: > > >>> > > > >>> > On Fri, 24 Jul 2026 at 15:06, Branko =C4=8Cibej <[email protected]= > > wrote: > > >>> >> > > >>> >> On 24. 7. 2026 13:57, Daniel Sahlberg wrote: > > >>> >> > > >>> >> Den fre 24 juli 2026 kl 13:30 skrev Branko =C4=8Cibej < > [email protected]>: > > >>> >>> > > >>> >>> On 24. 7. 2026 13:23, Branko =C4=8Cibej wrote: > > >>> >>> > > >>> >>> On 24. 7. 2026 13:20, Ivan Zhakov wrote: > > >>> >>> > > >>> >>> On Sat, 18 Jul 2026 at 18:27, Nathan Hartman < > [email protected]> wrote: > > >>> >>> > I'm glad there's a good bit of housecleaning going on around > here. > > >>> >>> > We've accumulated so much cruft over the years, it makes my > > >>> >>> > head spin. +1 to axe the ctypes bindings. > > >>> >>> > > >>> >>> Speaking of cleanups, another candidate comes to mind: the BDB > FS backend. I think now would be an appropriate time to remove it: > > >>> >>> > > >>> >>> > > >>> >>> > > >>> >>> We can't remove it before 2.0. It doesn't matter that it's > deprecated, it's part of the published 1.0 feature set. > > >>> >>> > > >>> >>> > > >>> >>> > > >>> >>> Just to be clear: the existence of the BDB back-end is > equivalent to a public API. We have many deprecated public APIs that we > can't remove for the same reason. > > >>> >>> > > >>> >>> -- Brane > > >>> >> > > >>> >> > > >>> > [...] > > >>> >> > > >>> >> This was discussed previously [1] and we even received a bug > report from FreeBSD that it doesn't work [2]. In the first link Yasuhito > Futatsuki was of the opposite opinion: the FS type is not a public API. I > tend to side with him but I could be convinced otherwise. > > >>> >> > > >>> >> As Yasuhito point out, SVN_FS_TYPE_BDB is a public API which we > need to keep. I didn't study details but I assume the code in > libsvn_fs_base only implement a filesystem API reachable via the fs loade= r > (this is way over my paygrade so please educate me if I misunderstand > something!). In that case, we won't break anyone's code so anything that > compiles under 1.14 will still compile under 1.16, but it it would not be > able to load the BDB FS backend (just as if you didn't compile it in the > first place). > > >>> >> > > >>> >> Of course - we would force anyone using a BDB backend to do a > dump-and-reload cycle as part of their upgrade. > > >>> >> > > >>> >> Provided my understanding above is correct, I'm in favour of > removing BDB. > > >>> >> > > >>> >> > > >>> >> > > >>> >> What do we do with older clients that link with libsvn_fs_base > when a library upgrade makes that go away? At the very least we'd have to > keep a stub that implements all exported functions, or we break the ABI. > > >>> >> > > >>> > > > >>> > As far as I see libsvn_fs_base functions are not publicly > exported. We have some BDB FS specific functions [1] in libsvn_fs, but th= ey > already stub to newer API and I agree we should keep them. > > >>> > > > >>> >> > > >>> >> In other words, even if we decide that removing BDB support isn'= t > an API violation, we can't do this in 1.15 because it's not a trivial > change of removing a directory and a couple lines from build.conf. > > >>> >> > > >>> > I agree and I don't think we should be doing anything about BDB i= n > 1.15 at this point (except for release notes). The proposed plan was abou= t > removing it in trunk / 1.16. > > >>> > > > >>> > [1]: > https://svn.apache.org/viewvc/subversion/trunk/subversion/libsvn_fs/fs-lo= ader.c?revision=3D1936528&pathrev=3D1936528&view=3Dmarkup#l786 > > >>> > > > >>> > -- > > >>> > Ivan Zhakov > > >>> > > >>> > > >>> This was discussed several times before, such as [1], [2]. > > >>> > > >>> Arguments in favor of removing BDB: > > >>> > > >>> (1) It has been deprecated for a long time. > > >>> > > >>> (2) AFAICT most devs are no longer testing BDB so maybe it works, o= r > > >>> maybe it bitrotted. > > >>> > > >>> (3) Last I checked, some kind of registration was needed to get the > > >>> official sources? > > >>> > > >>> (4) It might not have many (or any) users anymore. > > >>> > > >>> Arguments against removing it: > > >>> > > >>> (1) API guarantees... > > >>> > > >>> (2) Questions about what's involved in actually ripping it out. > > >>> > > >>> (3) Users with BDB-backed repos would need to dump and load them > > >>> BEFORE we remove it... > > >>> > > >>> ...and that means we need to give very clear, advance notice. > > >>> > > >>> So this is my suggestion: For now, let's start by making the > > >>> announcement together with the 1.15.0 release. I suggest to put the > > >>> announcement in the 1.15 release notes as well as creating a separa= te > > >>> news item for it. In the announcement, we won't commit to removing = it > > >>> by a specific date or release. We'll just convey that its removal i= s > > >>> being planned and may be imminent. > > >>> > > >>> After the announcement is made, Let's take as long as we need to > > >>> figure out the issues of API guarantees, how/what to remove, etc. > > >>> There's no rush to do the actual removal, but I think the > announcement > > >>> needs to give users as much time as possible. > > >>> > > >>> Proposed initial draft of announcement: > > >>> > > >>> [[[ > > >>> > > >>> IMPORTANT NOTICE: Subversion's Berkeley DB repository backend (BDB) > > >>> has been deprecated since Subversion 1.8 (13 years) and is planned > for > > >>> REMOVAL with a future Subversion release line. It will not be > possible > > >>> to access BDB-backed Subversion repositories after upgrading to a > > >>> Subversion release that no longer supports it. The Subversion 1.15.= x > > >>> release line may be the last minor release line to support the BDB > > >>> backend. > > >>> > > >>> Users who have BDB-backed Subversion repositories should upgrade th= em > > >>> to the newer FSFS backend as soon as possible to avoid disruption. > > >>> FSFS has been the default since Subversion 1.2 (21 years). This > > >>> requires to "dump" the repository using a Subversion release that > > >>> supports BDB and then to "load" the repository using any recent > > >>> Subversion release. For details, please see the FAQ entry "How do I > > >>> convert my repository from using BDB to FSFS or from FSFS to BDB?": > > >>> https://subversion.apache.org/faq.html#bdb-fsfs-convert > > >>> > > >>> ]]] > > >>> > > >>> Commenting on my own initial draft: > > >>> > > >>> Instead of sending them to the FAQ, maybe we'd want to write clear > > >>> backup/dump/load instructions directly in the announcement? > > >>> > > >>> Do we want to point out that the BDB backend might not be as well > > >>> tested as FSFS these days? > > >>> > > >>> WDYT? > > >> > > >> > > >> I think this is a good plan. I would prefer to keep the announcement > as short as possible and link to the FAQ, improving that if needed. > > >> > > >> I seem to remember that we at one point has talked about the lack of > testing - but that may just be on the mailing list..? I would probably no= t > mention it in the announcement - either something is supported or it is N= OT > supported. > > >> > > >> Cheers, > > >> Daniel > > >> > > >> > > >>> > > >>> > > >>> [1] "Berkely DB precompiled for Windows" > > >>> Posted to dev@ on 26 Sept 2024 > > >>> > https://lists.apache.org/thread/m4o5l9rntc98y6s8wrhsn329by9bhr19 > > >>> Continued here (disconnected in the archives): > > >>> > https://lists.apache.org/thread/99zo1wq003109p06dt1tcxyzd7cyyw32 > > >>> and here: > > >>> > https://lists.apache.org/thread/7plrvmhk3c0jvhkq1tfwds8j4tmlgslc > > >>> > > >>> [2] "Removing BDB support from 1.15" > > >>> Posted to dev@ on 18 Mar 2025 > > >>> > https://lists.apache.org/thread/kn0yw4f8p6tn2tzq4obr5r5v609g6g1m > > >>> > > >>> Cheers, > > >>> Nathan > > > > > > > > > It has been over four days and no further discussion on this thread s= o > I'd like to try making a summary. The last technical comment was made by > Ivan answering Brane's concerns about modifying public API ("libsvn_fs_ba= se > functions are not publicly exported. We have some BDB FS specific functio= ns > in libsvn_fs, but they already stub to newer API and I agree we should ke= ep > them"). > > > > > > @Branko =C4=8Cibej Does this satisfy your concerns? > > > > > > Anyone else? > > > > > > Cheers, > > > Daniel > > > > > > > > > > > > > Currently 'svnadmin help create' has this to say about the --fs-type ar= g: > > > > --fs-type ARG : type of repository: > > 'fsfs' (default), 'bdb' or 'fsx' > > CAUTION: FSX is for EXPERIMENTAL use only! > > > > Perhaps there should also be a "WARNING: BDB will be removed in an > > upcoming release?" or something to that effect? > > > I have drafted a notice of BDB's impending removal in the 1.15 release > notes > in r1936672. > > The text looks good to me! Thanks! --=20 Ivan Zhakov --000000000000b569550657b2f79e Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr">On Tue, 28 Jul 2026 at 18:22, Nathan Hart= man <<a href=3D"mailto:[email protected]">[email protected]= m</a>> wrote:</div><div class=3D"gmail_quote gmail_quote_container"><blo= ckquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left= :1px solid rgb(204,204,204);padding-left:1ex">On Tue, Jul 28, 2026 at 9:58= =E2=80=AFAM Nathan Hartman <<a href=3D"mailto:[email protected]" = target=3D"_blank">[email protected]</a>> wrote:<br> ><br> > On Tue, Jul 28, 2026 at 8:10=E2=80=AFAM Daniel Sahlberg<br> > <<a href=3D"mailto:[email protected]" target=3D"_blank">d= [email protected]</a>> wrote:<br> > ><br> > > Den fre 24 juli 2026 kl 21:03 skrev Daniel Sahlberg <<a href= =3D"mailto:[email protected]" target=3D"_blank">daniel.l.sahlberg= @gmail.com</a>>:<br> > >><br> > >> Den fre 24 juli 2026 kl 18:52 skrev Nathan Hartman <<a hre= f=3D"mailto:[email protected]" target=3D"_blank">hartman.nathan@gmai= l.com</a>>:<br> > >>><br> > >>> On Fri, Jul 24, 2026 at 11:52=E2=80=AFAM Ivan Zhakov <= <a href=3D"mailto:[email protected]" target=3D"_blank">[email protected]</a>>= ; wrote:<br> > >>> ><br> > >>> > On Fri, 24 Jul 2026 at 15:06, Branko =C4=8Cibej <= <a href=3D"mailto:[email protected]" target=3D"_blank">[email protected]</a>&= gt; wrote:<br> > >>> >><br> > >>> >> On 24. 7. 2026 13:57, Daniel Sahlberg wrote:<br> > >>> >><br> > >>> >> Den fre 24 juli 2026 kl 13:30 skrev Branko =C4= =8Cibej <<a href=3D"mailto:[email protected]" target=3D"_blank">brane@apa= che.org</a>>:<br> > >>> >>><br> > >>> >>> On 24. 7. 2026 13:23, Branko =C4=8Cibej wrot= e:<br> > >>> >>><br> > >>> >>> On 24. 7. 2026 13:20, Ivan Zhakov wrote:<br> > >>> >>><br> > >>> >>> On Sat, 18 Jul 2026 at 18:27, Nathan Hartman= <<a href=3D"mailto:[email protected]" target=3D"_blank">hartman.= [email protected]</a>> wrote:<br> > >>> >>> > I'm glad there's a good bit of = housecleaning going on around here.<br> > >>> >>> > We've accumulated so much cruft ove= r the years, it makes my<br> > >>> >>> > head spin. +1 to axe the ctypes binding= s.<br> > >>> >>><br> > >>> >>> Speaking of cleanups, another candidate come= s to mind: the BDB FS backend. I think now would be an appropriate time to = remove it:<br> > >>> >>><br> > >>> >>><br> > >>> >>><br> > >>> >>> We can't remove it before 2.0. It doesn&= #39;t matter that it's deprecated, it's part of the published 1.0 f= eature set.<br> > >>> >>><br> > >>> >>><br> > >>> >>><br> > >>> >>> Just to be clear: the existence of the BDB b= ack-end is equivalent to a public API. We have many deprecated public APIs = that we can't remove for the same reason.<br> > >>> >>><br> > >>> >>> -- Brane<br> > >>> >><br> > >>> >><br> > >>> > [...]<br> > >>> >><br> > >>> >> This was discussed previously [1] and we even re= ceived a bug report from FreeBSD that it doesn't work [2]. In the first= link Yasuhito Futatsuki was of the opposite opinion: the FS type is not a = public API. I tend to side with him but I could be convinced otherwise.<br> > >>> >><br> > >>> >> As Yasuhito point out, SVN_FS_TYPE_BDB is a publ= ic API which we need to keep. I didn't study details but I assume the c= ode in libsvn_fs_base only implement a filesystem API reachable via the fs = loader (this is way over my paygrade so please educate me if I misunderstan= d something!). In that case, we won't break anyone's code so anythi= ng that compiles under 1.14 will still compile under 1.16, but it it would = not be able to load the BDB FS backend (just as if you didn't compile i= t in the first place).<br> > >>> >><br> > >>> >> Of course - we would force anyone using a BDB ba= ckend to do a dump-and-reload cycle as part of their upgrade.<br> > >>> >><br> > >>> >> Provided my understanding above is correct, I= 9;m in favour of removing BDB.<br> > >>> >><br> > >>> >><br> > >>> >><br> > >>> >> What do we do with older clients that link with = libsvn_fs_base when a library upgrade makes that go away? At the very least= we'd have to keep a stub that implements all exported functions, or we= break the ABI.<br> > >>> >><br> > >>> ><br> > >>> > As far as I see libsvn_fs_base functions are not pub= licly exported. We have some BDB FS specific functions [1] in libsvn_fs, bu= t they already stub to newer API and I agree we should keep them.<br> > >>> ><br> > >>> >><br> > >>> >> In other words, even if we decide that removing = BDB support isn't an API violation, we can't do this in 1.15 becaus= e it's not a trivial change of removing a directory and a couple lines = from build.conf.<br> > >>> >><br> > >>> > I agree and I don't think we should be doing any= thing about BDB in 1.15 at this point (except for release notes). The propo= sed plan was about removing it in trunk / 1.16.<br> > >>> ><br> > >>> > [1]: <a href=3D"https://svn.apache.org/viewvc/subver= sion/trunk/subversion/libsvn_fs/fs-loader.c?revision=3D1936528&pathrev= =3D1936528&view=3Dmarkup#l786" rel=3D"noreferrer" target=3D"_blank">htt= ps://svn.apache.org/viewvc/subversion/trunk/subversion/libsvn_fs/fs-loader.= c?revision=3D1936528&pathrev=3D1936528&view=3Dmarkup#l786</a><br> > >>> ><br> > >>> > --<br> > >>> > Ivan Zhakov<br> > >>><br> > >>><br> > >>> This was discussed several times before, such as [1], [2]= .<br> > >>><br> > >>> Arguments in favor of removing BDB:<br> > >>><br> > >>> (1) It has been deprecated for a long time.<br> > >>><br> > >>> (2) AFAICT most devs are no longer testing BDB so maybe i= t works, or<br> > >>> maybe it bitrotted.<br> > >>><br> > >>> (3) Last I checked, some kind of registration was needed = to get the<br> > >>> official sources?<br> > >>><br> > >>> (4) It might not have many (or any) users anymore.<br> > >>><br> > >>> Arguments against removing it:<br> > >>><br> > >>> (1) API guarantees...<br> > >>><br> > >>> (2) Questions about what's involved in actually rippi= ng it out.<br> > >>><br> > >>> (3) Users with BDB-backed repos would need to dump and lo= ad them<br> > >>> BEFORE we remove it...<br> > >>><br> > >>> ...and that means we need to give very clear, advance not= ice.<br> > >>><br> > >>> So this is my suggestion: For now, let's start by mak= ing the<br> > >>> announcement together with the 1.15.0 release. I suggest = to put the<br> > >>> announcement in the 1.15 release notes as well as creatin= g a separate<br> > >>> news item for it. In the announcement, we won't commi= t to removing it<br> > >>> by a specific date or release. We'll just convey that= its removal is<br> > >>> being planned and may be imminent.<br> > >>><br> > >>> After the announcement is made, Let's take as long as= we need to<br> > >>> figure out the issues of API guarantees, how/what to remo= ve, etc.<br> > >>> There's no rush to do the actual removal, but I think= the announcement<br> > >>> needs to give users as much time as possible.<br> > >>><br> > >>> Proposed initial draft of announcement:<br> > >>><br> > >>> [[[<br> > >>><br> > >>> IMPORTANT NOTICE: Subversion's Berkeley DB repository= backend (BDB)<br> > >>> has been deprecated since Subversion 1.8 (13 years) and i= s planned for<br> > >>> REMOVAL with a future Subversion release line. It will no= t be possible<br> > >>> to access BDB-backed Subversion repositories after upgrad= ing to a<br> > >>> Subversion release that no longer supports it. The Subver= sion 1.15.x<br> > >>> release line may be the last minor release line to suppor= t the BDB<br> > >>> backend.<br> > >>><br> > >>> Users who have BDB-backed Subversion repositories should = upgrade them<br> > >>> to the newer FSFS backend as soon as possible to avoid di= sruption.<br> > >>> FSFS has been the default since Subversion 1.2 (21 years)= . This<br> > >>> requires to "dump" the repository using a Subve= rsion release that<br> > >>> supports BDB and then to "load" the repository = using any recent<br> > >>> Subversion release. For details, please see the FAQ entry= "How do I<br> > >>> convert my repository from using BDB to FSFS or from FSFS= to BDB?":<br> > >>> <a href=3D"https://subversion.apache.org/faq.html#bdb-fsf= s-convert" rel=3D"noreferrer" target=3D"_blank">https://subversion.apache.o= rg/faq.html#bdb-fsfs-convert</a><br> > >>><br> > >>> ]]]<br> > >>><br> > >>> Commenting on my own initial draft:<br> > >>><br> > >>> Instead of sending them to the FAQ, maybe we'd want t= o write clear<br> > >>> backup/dump/load instructions directly in the announcemen= t?<br> > >>><br> > >>> Do we want to point out that the BDB backend might not be= as well<br> > >>> tested as FSFS these days?<br> > >>><br> > >>> WDYT?<br> > >><br> > >><br> > >> I think this is a good plan. I would prefer to keep the annou= ncement as short as possible and link to the FAQ, improving that if needed.= <br> > >><br> > >> I seem to remember that we at one point has talked about the = lack of testing - but that may just be on the mailing list..? I would proba= bly not mention it in the announcement - either something is supported or i= t is NOT supported.<br> > >><br> > >> Cheers,<br> > >> Daniel<br> > >><br> > >><br> > >>><br> > >>><br> > >>> [1] "Berkely DB precompiled for Windows"<br> > >>>=C2=A0 =C2=A0 =C2=A0Posted to dev@ on 26 Sept 2024<br> > >>>=C2=A0 =C2=A0 =C2=A0 =C2=A0<a href=3D"https://lists.apache= .org/thread/m4o5l9rntc98y6s8wrhsn329by9bhr19" rel=3D"noreferrer" target=3D"= _blank">https://lists.apache.org/thread/m4o5l9rntc98y6s8wrhsn329by9bhr19</a= ><br> > >>>=C2=A0 =C2=A0 =C2=A0Continued here (disconnected in the ar= chives):<br> > >>>=C2=A0 =C2=A0 =C2=A0 =C2=A0<a href=3D"https://lists.apache= .org/thread/99zo1wq003109p06dt1tcxyzd7cyyw32" rel=3D"noreferrer" target=3D"= _blank">https://lists.apache.org/thread/99zo1wq003109p06dt1tcxyzd7cyyw32</a= ><br> > >>>=C2=A0 =C2=A0 =C2=A0and here:<br> > >>>=C2=A0 =C2=A0 =C2=A0 =C2=A0<a href=3D"https://lists.apache= .org/thread/7plrvmhk3c0jvhkq1tfwds8j4tmlgslc" rel=3D"noreferrer" target=3D"= _blank">https://lists.apache.org/thread/7plrvmhk3c0jvhkq1tfwds8j4tmlgslc</a= ><br> > >>><br> > >>> [2] "Removing BDB support from 1.15"<br> > >>>=C2=A0 =C2=A0 =C2=A0Posted to dev@ on 18 Mar 2025<br> > >>>=C2=A0 =C2=A0 =C2=A0 =C2=A0<a href=3D"https://lists.apache= .org/thread/kn0yw4f8p6tn2tzq4obr5r5v609g6g1m" rel=3D"noreferrer" target=3D"= _blank">https://lists.apache.org/thread/kn0yw4f8p6tn2tzq4obr5r5v609g6g1m</a= ><br> > >>><br> > >>> Cheers,<br> > >>> Nathan<br> > ><br> > ><br> > > It has been over four days and no further discussion on this thre= ad so I'd like to try making a summary. The last technical comment was = made by Ivan answering Brane's concerns about modifying public API (&qu= ot;libsvn_fs_base functions are not publicly exported. We have some BDB FS = specific functions in libsvn_fs, but they already stub to newer API and I a= gree we should keep them").<br> > ><br> > > @Branko =C4=8Cibej Does this satisfy your concerns?<br> > ><br> > > Anyone else?<br> > ><br> > > Cheers,<br> > > Daniel<br> > ><br> > ><br> > ><br> ><br> > Currently 'svnadmin help create' has this to say about the --f= s-type arg:<br> ><br> >=C2=A0 =C2=A0--fs-type ARG=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 : t= ype of repository:<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 'fsfs' (default), 'bdb' = or 'fsx'<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 CAUTION: FSX is for EXPERIMENTAL use onl= y!<br> ><br> > Perhaps there should also be a "WARNING: BDB will be removed in a= n<br> > upcoming release?" or something to that effect?<br> <br> <br> I have drafted a notice of BDB's impending removal in the 1.15 release = notes<br> in r1936672.<br> <br></blockquote><div><div class=3D"gmail_quote">The text <span class=3D"gm= ail-il"><span class=3D"gmail-il">looks</span></span> good to me!</div><br><= /div><div>Thanks!</div><div><br></div></div><span class=3D"gmail_signature_= prefix">-- </span><br><div dir=3D"ltr" class=3D"gmail_signature">Ivan Zhako= v</div></div> --000000000000b569550657b2f79e--