Re: API specification and library hell still not solved.

 
> --=-8AHHxopN8XpM5gEH6v/N
> Content-Type: text/plain
> Content-Transfer-Encoding: quoted-printable

Aargh, why that? :-)
 
> On Wed, 2003-08-20 at 10:20, Marco van de Voort wrote:
> > > > The API (headers) of Linux (kernel and *nix userland) is in C, and C =
> is
> > > > kind of problematic to convert to an arbitrary format (e.g. a differe=
> nt
> > > > language)..=20
> > >=20
> > > This is because the APIs are not generic APIs. They're C APIs.
> >=20
> > Cool. Don't care. Maybe I was mistaken. Where are the real API's then?
> 
> They are the real APIs. Unix was written in C. (Once it was finalised,
> anyway, the first coding on the PDP-7 was in assembly).

I'm well aware of the history of Unix. But if the C APIs are not for generic
use, which are?
 
> If you don't like that, perhaps you should have paid more attention in C
> class so you could port them to the language of your choice.

I choose not to use C. Has nothing to do with paying attention in class or
not. Please try to remain somewhat constructive.
 
> > But those runtimes still have to connect to something OS dependant
> > somewhere, and whatever that is (syscalls/kernel, like in the FPC
> > case, or libc), you'll need headers. Which means header conversion
> > =09
> > For our libc translation, see
> > http://www.freepascal.org/cgi-bin/viewcvs.cgi/fpc/packages/base/libc/
> > (+/- 800 kb)
> >
> > Can you imagine checking that for each minor update of libc?=20
> 
> If you were writing a compiler and didn't verify each minor update of
> libc, I personally wouldn't be remotely interested in using your
> language.

That's your choice. And I would like to check it, however, there is no
system to do so coveniently.
 
> This is called "work".

More like slave labour.
 
> > Can you imagine what happens if you have 60-70 packages besides several
> > incarnations of libc, and say 20 platform-architecture combinations in us=
> e?

> No. I stick to one incarnation of libc per language.

That's not the case. FreeBSD already has a different one, so does Solaris.
 
> IX
> > > Specification - which is the document glibc aims for conformance with, =
> and
> > > provide your declarations of what's there in whichever language you ple=
> ase.
> >=20
> > Won't work. Those specifications are at the C level. It doesn't say anyth=
> ing
> > about the actual contents. More over it is not just the OS, but also pkgs=
> ,
> > BSD libc's etc.
> 
> As the "C level" is virtually synonymous with machine level,

That's not true. If I specify a struct, e.g. the C language specifies
nothing about alignment, and only the minimal sizes and for basic types.
(and that some types are equal or larger than others)
So there's a lot compiler implementation defined.

See also the part in the previous msg about the header system being useful to use
different C compilers. Or even ancient ones (e.g. OpenBSD used a pretty old
one until quite recently because they were still a.out).

And the Borland Pascal dialects can map nearly 1:1 to that nearly. So that's
not the problem. If I could decode the C syntax automatically, I could
convert it to an interface. The problem is that I can't.
 
> ely
> > > ignore the kernel C headers and use their own with matching definitions=
>  of
> > > userspace-visible structures and macros.
> >=20
> > This is pretty much what we do now. It's however simply too much
> > (and unnecessary) work.
> 
> Ignoring the API which is in a state of flux and concentrating on the
> static one is too much work?

Manually doing it, while it could be done automatically. The data is there,
it is just not accessable.
 
You only say "do the work", you still have to come up with the first
argument why my scheme wouldn't work.

> > > Not at all. The job of a configure script is to determine the differenc=
> es
> > > between platforms - bought about by varying levels of (in)compatibility=
>  between
> > > deceptively similar UNIX systems. (For example, some systems have a
> > > <alloca.h>, some don't). The tests configure perform are actually very
> > > simple indeed. The two most common are:
> >=20
> > Sure. But why are the tests on _TYPING_, function existance etc done on
> > the binaries instead of headers? =20
> 
> Assuming you mean "Why are the header files tested and not the
> binaries?"

No, I meant why are test programs compiled, instead of parsing the headers.

Meanwhile I thought abotu it, and there is one reason, the test program
tests a compiler-header combination, and predefined values in the compiler
(or dependant on compiler version) can have effect on the headers.  

> > > > automatic conversion.
> > >=20
> > > Actually, the Microsoft angle is apalling. It offers no alignment with =
> any
> > > standards whatsoever in its SDK.
> >=20
> > [strip M$ IDL version being non standard]
> >=20
> > So? I'm not handing out cookies for standards compliance, I simply want
> > access to the OS with as little trouble as possible, as a user of that OS=
> ,
> > starting from what the OS vendor ( in the linux case GNU userland parts,
> > Linux kernel, and other packages, in Microsoft's case the Platform SDK, a=
> nd
> > possibly other SDKs) provides.
> 
> Then perhaps you should use an OS that doesn't try and be so inclusive.

Or maybe the OS should evolve? Some commercial OSes already used a scheme
like this. IIRC also QT has some form of "master" headers from which all
different language headers are formed (by Perl)

> Microsoft let you code the Microsoft way, which provides the amount of
> trouble you are willing to deal with, so perhaps you should be using
> that platform.

This has nothing to do with Microsoft, except for the comparison. It is just
common sense.

Again: Come with real arguments except some misplaced C conservatism, or name real
disadvantages of the solution if you think it is no good.
 
> > Microsoft provides that better than Linux currently, from my point of vie=
> w.
> > Microsoft has other evils, but those are not under discussion right now.
> > This is about improving *NIX, and the header situation could do with some
> > improvements.
> 
> I haven't had a single problem with it. I code in a number of languages,
> including C, and they all deal with this in a perfectly acceptable way.

And how did the non-gcc other compilers deal with this problem? Maybe they
simply also did a lot of useless work for each version, just like we do now
(since SLS times) That doesn't mean that that's the ideal situation.

Simply saying "you shouldn't do that, or you should use C" is too
simplistic. That might get you a bit of karma on Slashdot, but I thought this
list was about serious brainstorming about possible Linux deficiencies.
 
> > >=20
> > > Solution: Keep your custom language's runtime library separate from the
> > > system C library.=20
> >=20
> > In 5 years working on the FPC compiler, you are the first one to agree wi=
> th
> > me with that. The usual reaction is "Wow, you don't work via libc?"
> 
> Make me the second.

Added a notch.
 
> > However for FPC that situation is getting harder. Libc is starting to get
> > more and more functionality integrated, and FPC is simply more advanced, =
> and
> > the avg programs needs to link to other C libs anyway. (see e.g.
> > lazarus.freepascal.org, our RAD as example)
> 
> Then your language should have the hooks necessary to do so. Perl does.

Define "hooks"?

Anyway that is a different situation, Perl needs kludges because it doesn't
translate to C 1:1. Pascal (at least the Borland ones) does pretty much.

Hooks still need definition, and that definition I must get from the
interface specification, and we are back at the headers.

> > > Each language is and should be independent, and each is
> > > capable of making system calls should it need to.
> >=20
> > Converting Unistd.h is not a problem, I could do that between dinner and
> > desert.
> 
> 1 down, X to go.

Believe me. I've done it. It's not doable/sustainable long term.
 
> > > probably be using C in the first place.
> >=20
> > That is ridiculous. Why should that be necessary? OSes have a history of
> > being written in C, yet zillions of other languages somehow interface to =
> it.
> >
> > I'm sorry, but that's simply Unix/C zealotry. I don't want to use the C a=
> pi,
> > I simply have too, there is no other userland. If you want to split up li=
> bc
> > into libuser(land) and libc, you have my blessing.
> 
> If you don't want to use the C api, don't use Unix. 

Why should I? Because you say so? Please provide reasons.

> The two have been
> intertwined since the beginning and are likely to remain so for a
> considerable time to come.

Nothing in my proposal ends that. Also most commercial Unix OSes are
actually more foreign language friendly than Linux. That's because nearly
each one came with a Fortran and/or Pascal compiler, besides the system C
compiler (sometimes the userland C compiler aws even different from the
system (kernel) one)

Please read the original proposal again. Nothing changes for the C side.
except maybe for the better (better non-system C compiler support)

The only thing that changes is that the C headers get generated from a
central source that is somewhat better parsable.
 
> > Moreover it is not simply a question of C vs Pascal. While less severe, I=
> 'm
> > in the same situation if I want to use a C compiler that is not the same
> > (family as the) system C compiler. I might need to add directives, and mi=
> ght
> > not be able to process e.g. certain gcc extensions.
> 
> If you want to use a different C compiler, you need different C
> libraries. Somehow that sounds to me like logic.

No. Besides being C runtime, libc is also the userland part of the OS
nowadays, and that trend is still getting more obvious each day.

Besides, glibc is afaik gcc specific, and wouldn't even compile on the avg
compiler. 
 
> >=20
> > I can't be on each kernel hacker, libc, userland tools etc maillist  for
> > each and every one of them? And then also follow a list for each
> > architecture-platform combination and the package (ports) system for all
> > distributions etc etc.
> 
> So you want to support 10 OSs and 4 architectures but you don't want to
> know anything about them?

I don't want to process details that can be done automatically, and that for
each version again.

Give me one good reason why I should, or at least why it would be beneficial
to Linux !
 
> I'm glad I never wasted my time with Pascal.
> 
> [... snip library hell ...]
> 
> I'm sorry if this sounds flamewarish, 

It does. On a horrible level. You obviously haven't actually given the
proposals a chance, or even thought openminded about it. Otherwise you would
have had better reasons than "Unix is C is Unix, go use Windows"

> but you give the impression of
> wanting to code (wanting to code a compiler, no less) without bothering
> to try and understand the system[s] you are coding for.

I'm already coding, I've been doing this for the past 5-8 years. (5-6 years
FPC core 2-3 years mere user before that). I had alraedy started using free
unices with 4.3BSD, and remained to do so ever since. (or maybe aversion
early, can't remember exactly)

I think I know enough about these matters to be at least receive arguing
based on solid reasons instead of flames.

The FPC project is over 10 years old, in a lot of linux distro's (Debian, Suse
already for half a decade and I heard it will be in the next RH too, finally) and
BSDs (FreeBSD, OpenBSD,NetBSD, Darwin), Solaris, BeOS (glibc user). The
linux port itself dates from '95, kernel 1.1.x

The project is well over 100 MB source (half of it header translations btw),
selfhosting (only uses ld as external tool) has a GTK RAD, can compile
nearly the entire Delphi dialect.

> That, I'm afraid, sounds incredibly dumb.

I don't want to check every dumb detail every time a version number changes.
As said we only support 4 architectures. I'd rather expand that then doing
the same header conversion each time over and over again.