Re: Statically building when curl is present

Thanks Denis!  Replies below...

On Thu, Jun 15 2023, Denis Corbin wrote:

> On 15/06/2023 15:09, John Goerzen wrote:
> this comes from glibc, this is just a warning and has for consequence a
> duplication of glibc in memory when running the statically linked executable.
> For that reason, when providing dar_static binaries (which are distro
> independents), I got to replaced glibc by "musl", I think this was the during
> investigation where we probably hit a bug in glibc see the discussion with
> Martin Michel having this subject

I think it should be fairly easily to build the static version against
musl in Debian.  Debian does make musl available.

> I kept trace of this work asa script you can find in source code:
> misc/dar_static_builder_with_musl_voidlinux.bash

It looks like you have solved the curl problem there by building curl
yourself with most of the optional features disabled.  That makes sense,
but I won't be able to do that in the Debian context.

> Yes, I remember this. Hopefully libcurl supports alternatives. This script
> should give you some hints like relying on gnutls rather openssl which has a
> much more dependencies, from which the BrotliDecoderVersion and similar symbols
> are expected...

Yes, Curl supports 3 different TLS libraries: OpenSSL, gnutls, and NSS.
I looked into gnutls already; it is only marginally better, but since
krb5 is a dependency of Curl, not OpenSSL, it particularly doesn't fix
that problem.

> so you miss a static libgssapi_krb5 (kerberos) or miss linking with it, which
> library dar does not use by the way... but which is very probably needed by a
> dependency below openssl I guess.

Below curl, but yes, that's the situation.  There is no static
libgssapi_krb5 available.

> I guess they rely on what I provide at
> https://dar.edrusb.org/dar.linux.free.fr/Releases/Dar_static/

I can't do that in Debian, for a couple of reasons:

1) policy requiring binaries shipped by Debian must be built by Debian,

and 2) Debian supports a dozen archs beside x86_64.

>> My best guess is that maybe they're on
>> platforms where curl itself was compiled with fewer features.
>
> Well, if you build a static binary/library you can temporarily build
> dependencies that even may be of a different version than the corresponding
> packaged dynamically libraries available on the distro, at the end of the
> process you get the static binary (which you can eventually strip to reduce its
> size), then the rest goes to trash.

For lots of complex reasons, having a stripped-down curl is probably
impractical but I could investigate I suppose.  I guess a static rb5
may also be a possibility.  I previously got the static librsync in
Debian, but I don't know how easy that will be for krb5.

>> Anyhow, then the question becomes: how to resolve this?
>> As far as I can tell, the only solution is to not link dar_static with
>> curl.  I don't see that option in configure, though; it's all or
>> nothing with that one.
>
> this is an option. But try building a static flavor of libcurl based on gnutls
> rather openssh (see the script I mentionned) it should be a bit more simple I
> guess...

I think there are two ways forward:

1) The dar build system is adapted to support building the
dynamically-linked executable with curl, and the static without;

or 2) During the build, I copy the source tree and build the entire
thing twice, with different configure parameters.

I have option 2 already working, I think (haven't tested it yet).
Option 1 would be a lot more elegant on my end but possibly not on
yours, especially given the config.h(pp) problem I mention below.

Both have the potential for causing problems for things that use libdar,
as I note libdar_config.h has #define LIBDAR_HAS_CURL_CURL_H and
LIBDAR_HAS_LIBCURL.  If the libdar.so is built with different configure
settings than libdar.a, the headers will be incorrect for one of them.

On the other hand, there are presently no users of libdar.a in Debian
other than dar_static itself, and I think it highly unlikely that there
ever would be, so perhaps I could just hand-wave that problem away.

> Maybe also I could be good to ask some Debian Guru, if this is acceptable in
> regard the Debian policy, to have inside dar_static a gnutls library compiled
> with slightly different options gnutls vs openssl, but using the same source
> code, same version as the official one...

Generally no (the problem would be with curl, not gnutls)...  I would
need to get the curl source package to build a different set of library
and dev packages.

>> BTW on a totally different topic, just wanted to verify you saw
>> https://github.com/Edrusb/DAR/issues/50
>
> Yes, and I think I have answered you by email... let me double check... I will
> update the github issue... :)

Perfect, thank you!  That's one more bug closed.

- John