RE: [serf] r2489 committed - In preparation of serf 1.4.0, remove the get_remaining function from t...

> -----Original Message-----
> From: Ivan Zhakov [mailto:[email protected]]
> Sent: zaterdag 12 september 2015 18:30
> To: Greg Stein <[email protected]>
> Cc: [email protected]
> Subject: Re: [serf-dev] [serf] r2489 committed - In preparation of =
serf 1.4.0,
> remove the get_remaining function from t...
>=20
> On 12 September 2015 at 19:08, Greg Stein <[email protected]> wrote:
> >
> > On Sep 12, 2015 11:01 AM, "Ivan Zhakov" <[email protected]> wrote:
> >>
> >> On 12 September 2015 at 15:08, Greg Stein <[email protected]> wrote:
> >> > [redirected to dev@serf]
> >> >
> >> > On Mon, Apr 6, 2015 at 8:32 AM, Bert Huijben <[email protected]> =
wrote:
> >> >
> >> >> > -----Original Message-----
> >> >> > From: [email protected] [mailto:serf-
> [email protected]] On
> >> >> > Behalf Of [email protected]
> >> >> > Sent: maandag 6 april 2015 11:25
> >> >> > To: [email protected]
> >> >> > Subject: [serf-dev] [serf] r2489 committed - In preparation of =
serf
> >> >> 1.4.0,
> >> >> > remove the get_remaining function from t...
> >> >> >
> >> >> > Revision: 2489
> >> >> > Author:   lieven.govaerts
> >> >> > Date:     Mon Apr  6 09:24:18 2015 UTC
> >> >> > Log:      In preparation of serf 1.4.0, remove the =
get_remaining
> >> >> > function
> >> >> > from the
> >> >> > bucket API.
> >> >> >
> >> >> > This reverts most of r2008, r2009, r2010 and r2198. From r2008 =
I kept
> >> >> > the
> >> >> > read_bucket_v2 function, which is needed for set_config.
> >> >>
> >> >> If we still keep the read_bucket_v2 feature, what is the reason =
for
> >> >> just
> >> >> removing get_remaining?
> >> >>
> >> >>
> >> >> We still have to handle the linkage problems if we use the
> >> >> read_bucket_v2() system for versioning, but we leave out the =
tests that
> >> >> were added to find problems caused by different function =
pointers.
> >> >>
> >> >>
> >> >> The reason we see errors on this feature on several platforms is =
just
> >> >> this
> >> >> linkage problem, while the get_remaining code itself is pretty =
stable
> >> >> as
> >> >> far as I can tell. I'm guessing that the reason for removing the =
code
> >> >> is
> >> >> that we see this errors.
> >> >>
> >> >>
> >> >> Our scons scripts make us link both the static library and =
shared
> >> >> library
> >> >> versions of the same functions, and then a simple pointer =
comparison on
> >> >> a
> >> >> function pointer isn't going to work as expected.
> >> >>
> >> >>
> >> >> And with a shared library build, there are other possible =
problems:
> >> >> sometimes you get a pointer to a wrapping function... or the =
pointer
> >> >> value
> >> >> is not guaranteed because the code might be moved under some
> >> >> circumstances
> >> >> (unload+load of library)
> >> >>
> >> >
> >> > So you're saying that a pointer to a specific function might have =
*two*
> >> > values under Windows? Within a single executable? That A could =
see a
> >> > different value from B?
> >> >
> >> This is not Windows specific problem, the original problem report =
was on
> >> Linux:
> >> "Buckets v2 check doesn't work on Linux in the test suite
> >> (test_aggregate_buckets fails)" [1]
> >>
> >> Bert commented  that serf on Windows had similiar problems, but I
> >> don't know details.
> >> [1] https://issues.apache.org/jira/browse/SERF-134
> >
> > That JIRA indicates there are multiple copies of the function, which =
is
> > clearly wrong.
> I don't know details, but it seems related how linker, loader and
> relocation works on Linux:
> =
http://reverseengineering.stackexchange.com/questions/1992/what-is-plt-go=
t

I just experimented a bit with dll function exports with Microsoft =
compilers.

If I use the pattern we commonly use: .def file for exports and then =
using the .lib file and normal function definitions in .h files the =
function pointers are *not* equal.

But if you annotate the functions with __declspec(dllimport) on the =
import side then the function pointers *are* equal.

For the export side it does not matter if you use .def or =
__declspec(dllexport).

An import without an annotation uses the stub

I think =
http://stackoverflow.com/questions/22766540/semantics-of-imp-symbols =
provides more info.



Note that using whole program optimization ("link time code generation") =
looks through these stubs and returns the actual function pointers in =
all cases.


Summarizing:
* With our current code we can't trust pointer equality when using a DLL =
build on Windows
* Marking (all?) interesting functions with __declspec(dllimport) will =
fix the problem
(* Users can avoid the problem via LTCG in release mode.)


Using LTCG even made 4 different functions with just the same printf() =
call inside all have the same address!

	Bert