RE: [NeoStats-Devel] ADNS and libevent

Justin Hammond wrote:
> > Justin Hammond wrote:
> > > Some *other* people were using our implementation of adns with 
> > > libevent... I added that so they could update the fd 
> callback  for 
> > > them...
> > 
> > And I am supposed to know this how? 
> > 
> > I did announce my intentions prior to making the changes. 
> The project 
> > is NeoStats, not library repository for libevent so I assume that 
> > anything within the NeoStats source tree can be modified 
> for NeoStats 
> > use. If you wish to provide a library repsitory for libevent or for 
> > any other prupose, why not create a separate project for 
> these "other" 
> > people to rely on that will never be affected by NeoStats specific 
> > changes? This would have the additional advantage of 
> allowing it to be 
> > properly packaged as a standalone download.
> > 
> 
> And as mentioned before, I've had hardware problems, hence my 
> lack of work in the last week. I've had intermitant access to 
> the internet, and honestly, I've been concentrating on paying 
> customers when I have been down. 
> 
> I guess as previously mentioned on this list with regards to 
> the whole library debates, I'm a proponent of the GPL in 
> effect of "giving back" to the community. 

The change does not stop you giving back a libevent aware version of the
library. As mentioned previously it is still in the socket branch. 

> Although the 
> development might not be active on the library, its still 
> actively supported by the authors 
> (http://www.chiark.greenend.org.uk/pipermail/adns-discuss/2005
> /001094.html
> as a example) so at any time, there might be updates to it. 
> Last year there was discussions about adding ipv6 support to 
> ADNS. 

3 messages this year, none last year, 40ish in 2003 1 in 2002, 20ish in 2001
and in all this time no updates. The ipv6 discussion was not last year but
was in 2003. This was a ressurection of a discussion from 2000. Both of
these lead to nothing. 

> It never materialized into anything yet, but if someone 
> were to write it, we now have *more* work in updating our 
> ADNS to support ipv6 if we ever need/want to support ipv6 in 
> NeoStats..... 

Actually we do not. We write the main code once in the abstraction layer and
all of NeoStats benefits. If they ever release an IPv6 version, NeoStats
would need to support it before it was of any use to us in any case. The
change makes it easier to look into IPv6 support since we can investigate it
at a core level and it propogate automatically to the absorbed libraries.
Without absorbtion, we must develop for the core and other libraries
resulting in more work.

Since we have a copy of ADNS in the 2.5.x branch and the socket branch
version, we can easily create a diff against any update and what we started
with and apply the results to the integrated version of ADNS since the code
is easily identified. Due to the changes for libevent and for Win32, an ADNS
update would never have been a simple drop in update so the amount of work
is comparable regardless of whether ADNS was integrated or a library.

> The last thing I want to do now is go out and start 
> maintaining separate copies of libraries as a separate 
> projects. 

I did not suggest that. I suggested you create a repository into which such
things could be put. That way instead of having to dig through neostats or
the socket branch, people could get just adns from the library respository.
There is no requirement on anyone to maintain it but if you want it
available to others, it seems a more sensible proposition than keeping ADNS
as a library within NeoStats just so other people can download it. 

Your email of Sunday implied there were hordes of angry users of portions of
NeoStats baying for your blood, the suggestion was to show there was no
reason for concern by or about "other" users and a way to provide a
permanent reference for your libevent aware ADNS library.

I did not make the change to annoy you, these "others", to circumvent your
own thoughts on GPL, or for the sake of it but solely to benefit NeoStats
and reduce the workload of getting it working fully on Win32. 

ISTR that libconfuse is to be integrated into NeoStats. That the original
dotconf library was integrated into NeoStats. Portions of kazlib for lists
and hashes were intergrated into NeoStats. Keeper was modified for NeoStats
and became intrinsically dependent on it. There are likely other examples
but none of these were of any great concern for "giving back" or for "future
updates" or for "others" so I do not see why ADNS represents such a huge
problem for multiple reasons whilst these others do not. The changes being
made only benefit NeoStats so there is noone who could benefit from them
other than NeoStats. 

> Agreed that the project is NeoStats, but I'm honestly 
> scratching my head as to the benefit of porting ADNS to use 
> our abstraction layer? I don't believe we had any porting 
> issues with ADNS on either Win32 or *nix platforms (we did 
> have two libraries... But your initial merge of these two 
> libraries sorted that out without having to use our 
> abstraction/memory management functions correct?)

ADNS for win32 had a number of problems. The use of and hacks to use errno
were often erroneous. It had some unix api additions that our own
abstraction layer provides yet were not necessarily compatible with ours or
the conflicting ones provided in curl and it made sense to maintain only one
set rather than multiple sets of such calls. It has intermmittent use of the
correct methods of allocating and freeing memory so moving all memory
functions to the abstraction layer addresses this. As has been mentioned
many times, the intent is to remove crt calls from NeoStats on win32 and
replace them with native win32 calls. The abstraction layer allows this to
be done once and in one place rather than having to do a bit in neostats and
bits in various libraries. 

To keep ADNS as a library would have required a lot of work to get it 100%
compatible with windows, most of which would be duplication of work already
done or to be done in the abstraction layer so the integration is future
proofing and reducing code maintainance.

> Maybe its just more from the point of view that I see this as 
> fixing something that wasn't broken.... :) My original change 
> to ADNS was pretty minimal, and if you didn't use libevent or 
> some other io abstraction, it had no effect on you... 

The problem is that it was broken, just not obviously so since it compiled
and ran but that was all the original library was intended to do in order
for Win32 development to proceed. ADNS for win32 in it's original form had
the problems mentioned above and would have ultimately required additional
work to get working correctly that can be avoided by using the abstraction
layer. I have no intention of becoming a maintainer for various libraries
and integration allows the fixes to be made without being restricted to
keeping ADNS a working and distributable library or having multiple copies
of similar code and the maintainance overhead that goes with it.

This you may remember is one of my reasons for recommending we absorb
libevent since there are areas of it that will not work for Win32 and these
could be passed to the abstraction layer rather than have to maintain
multiple code paths. I still maintain this position wrt libevent but you
have not commented either way on the proposal and I do not know the current
status of libevent so have not looked any more into it recently.

> Now we 
> might as well not call it ADNS anymore. 

I don't care what it is or isn't called and consider it's name irrelevent.
All files that were integrated have the adns prefix to identify the portions
derived from the library but since it would never be linked to anything
other than NeoStats, it is a mere subsystem based on ADNS. 

Anyway, since the subject is current, I think now is a good time to finalise
a decision on other libraries:

1) libevent: do we go with integration or keep it as a library we have to
fix if we don't upgrade and if we do. As stated previously I favour
integration so that it can use the abstration layer for those areas that are
incompatible and removes the restrictions the library creates such as the
need for dummy functions purely to avoid a segfault. The logging system of
course could be dumped completely and replaced with the core one. As with
adns, I would suggest a naming convention so we can identify the core
libevent code in the event we ever wanted to take updates from the authors
to the actual libevent systems that remain.

2) prce: I guess it needs to be updated to a more current version as a first
step. Is there any reason we could not jump to 5.0? I would prefer it to be
intregrated so that we can resolve the current overhead with win32 on
modules having to redirect the memory calls where it is used since we could
simply call the neostats memory system directly from a slightly modified
version. Integration also resolves the issue that automake has again created
with the change in link of getting pcre linked to the core as a single link
rather than to each module which uses it generating a local copy.
Modifications I see as being useful are to call the core level os string and
memory abstraction functions. As such they should not prove so intrusive as
to prohibit the application of any future updates or fixes to it despite it
being an integral part of NeoStats rather than a library. 


Mark.