Re: Re: [UnixOS2] libc

On Tue, Jun 17, 2003 at 06:49:52PM +0200, Stefan Neis wrote:
> On Tue, 17 Jun 2003, Adrian Gschwend wrote:
[...]
> I like the asynchronity (??) of mail. Everyone can read/reply when he can
> spare some minutes. ;-)

I prefer mailinglists over IRC as well for the reason of asynchronicity.

> > <achimha> well, our libc is done to fulfill our needs
> > <achimha> it is incompatible with emx (note by ktk: due to licensing
> > problems with EMX)
> 
> Huh? Which ones? I always had the impression that EMX's licence is rather
> permissive...

The old GPL debate. GPL is not free, it is rather restricted. Too restricted
to be acceptable for people who need tailored version of the software.

> > <achimha> our GCC is targeted to be a VAC/Watcom replacement not a Unix
> > porting environment
> > <achimha> just wait for our first distribution then we can discuss how
> > to combine the various efforts
> 
> To be honest, that doesn't sound to promising from the porting point of
> view. There are some things that _have_ to be deep inside libc, otherwise
> e.g. closing sockets by close instead of closesocket or pthread support
> won't work... :-(

I don't see a problem so far, unless "VAC/Watcom" compatibility is meant
in a way to copy all of their quirks and bugs and anachronisms one by one.
In such a case one would have to separate close and closesocket, for instance.
OTOH, Unix compatibility also means to copy numerous broken concepts of
the Unix API which are better solved in the OS/2 API. EMX tried some
balance there with the effect that some Unix things work, and some OS/2
things work, and special cases don't, in both areas.

> > For Unixos2-only people: Henry proposed to create a CVS for libc, now
> > the question is what do we want to take as base (EMX, LIBEMU and
> > Innotek-GCC libc is in dicussion, there is also one from IBM btw)
> 
> I think we should start with EMX, add Posix/2, some link support,
> pthreads, libintl and maybe some other nice little extensions and then
> start making it all work together and fixing bugs (e.g. pthread library
> and library for symlink support (via EA) both modify certain functions
> for accessing files - BTW, large file support is the third candidate which
> "wants" specific changes in that same area).
> 
> Whether the CVS is rooted at netlabs or sourceforge (we could start by
> abusing the Posix/2 CVS) is rather irrelevant to me, however it would be
> great to automatically send all CVS commits (as a diff) to some dedicated
> mailing list (I know that wxWindows did such a thing also while the CVS
> was hosted by sourceforge, so it's possible to set it up like this even
> there - but I don't know, how to do it), so whoever is interested has a
> chance to check things easily.

Meanwhile, with rather limited time, I second that fully.

> The biggest problem for starting however is probably the build system:
> EMX libs expect dmake, IIRC. Posix/2 wants some GNUmake, pthreads want
> I don't know what etc. Unifying this and "porting" the Makefiles to some
> current GNUmake is going to require much time for not much to gain... :-(

I had already started to move EMX to GNUmake (don't ask me which version,
I am typically off the recent porting attempt), but eventually messed this
up by trying, at the same time, to improve the EMX environment, e.g. by
throwing out DOSisms and the syscall interface through the INT21 like
register calling mechanism. What make the dmakefile build mechanism
complicated was the IMHO too complex interwoven system of dependencies
in order to build the various variants of linraries from a single source
tree. It needs some time to untangle this, but this is something that has
to be done before any further work.

> Ideally, we would also stay backward compatible, which implies keeping the
> right ordering for the already existing symbols in the DLL's. (though I'd
> suggest to make the import libraries for the new DLL's using "import by

You'll have to give up full compatibility eventually, no doubt about
this. Leave the old EMX*.DLLs in the system, and name the new ones EMU*.DLL
or anything else, and don't attempt to keep old code where you have better
new code. You do know that if you want to integrate libemu, you'll get a
clash on certain APIs. And then there is no need to keep the DLL ordinals
consistent with old EMX. They may be compatible in the first release, but
will diverge sooner or later. So might it right from the beginning (means:
name imports).

> name"). OTOH, I _suppose_ large file support if put into the "normal" C
> routines is going to cause incompatibilities anyway (file positions no

You can move file I/O into a separate DLL, and provide large file I/O
by DosLoadModule where present, and normal I/O where not. This is a similar
thing like EM's TCP/IP support (in those days it was not clear whether
every OS/2 system had SO32DLL.DLL, now you would rely on this).

> longer fit into an int, do they?), so maybe it would even be a good idea
> to be intentionally incompatible (old EMX DLL's handle small files, in the
> new runtime, everything would be large file enabled - Hmm, I'll have to
> get a new harddisk to create a large enough JFS partition for testing...).
> 
> > It would be nice if we really could discuss that all together and agree
> > on a sollution that is fine for everyone and allows enhancements in the
> > future (no dead end).
> 
> What I don't know at all is, how this could fit into Holger's long term
> plans. Ideally, replacing a couple of low level C routines by his more
> complete/efficient implementations - once they are ready - would be
> sufficient, but for this,  it would be a big plus to have a rather
> detailed list of what functions those are going to be, I suppose.

The API I am targeting is the man(2) API. This means there should be still
a separation of man(2) and man(3). This is more or less already implemented
in EMX, however with the mentioned syscall interface.

> E.g. an improved link implementation is going to affect some other file
> related functions ((f)open, (f)stat, ...), but if we know this in advance,
> we can have some internal function like __dereference_link that's called
> by all those, so the changes from a temporary E/A based link emulation to
> "real" links are minimized...

Look at the way it is done in "real" Unix. If the "link" code touches
any stream-I/O function, e.g. fstat() or fopen(), it is doing something
wrong. The link code should be based on lstat()/stat() and open(), and
in particular as close as possible to DosOpen and friends, because it
is very likely that a future DosOpen will already understand and handle
links. Implementing it in this way is IMHO the main challenge of 
cleaning up EMX code. And don't worry about EA-based vs. "real" links;
once the symlink code is stabilized, there is a way to do the same in
kernel mode (with the side effect that these EAs become restricted
and protected unless handled by the correct functions).

Holger