Re: ideas for returning memory to the OS

Enrico Weigelt wrote:
>> You mean in terms of codebase size?
> 
> Yes, and also memory consumption and required development/maintenance
> resources.

Sorry, at least the second of those doesn't follow from what you said? 
Do you have data to back it up?

Put another way, I do have data that shows that generic system libraries 
use more memory than custom-written code (that provides just the small 
subset of functionality that the app needs).  The only issue is how much 
less, how small a subset is needed, etc.  If you need a good chunk of a 
library, there's no point in not just using it.

>> Part of the problem, you see, is that OS libraries have a tendency to be broken. 
> 
> Right. Then try to fix these libraries

Some of the library maintainers aren't very interested in accepting 
help.  msvcrt was mentioned, for example.  But there are plenty of 
examples on Linux too, usually under the guise of "that's not what it's 
designed to do" or "it's good enough to run GNOME" or some such.

> (within their jobs) or tell the world about the broken crap.

The world knows.  Now what?  The world knowing doesn't make broken code run.

> If you try to work around evrybody else's crap, you'll never get finished.

And if you don't, you'll never work right and hence never ship.  Sounds 
like we should all pack up and go home.

> And once one of the bugs is fixed, you have to take care of disabling or
> removing the workaround (since it will most likely break then, again)

You're making the assumption that the behavior in question is (1) a bug 
and (2) will be fixed.  If the behavior is mandated by some standard 
(whether broken or not), it won't be considered a bug.  See POSIX.  Even 
if it's considered a bug there's no guarantee that it'll be fixed 
quickly, or at all.  See the Apache default MIME type story, for example.

> The trick in cross-platform development is not caring about every single
> difference, but abstraction: build upon interfaces which are DEFINED to
> be platform independent (and maybe help implementing these within their
> dedicated projects).

Then you need to implement those abstractions.  Which means either 
per-platform code, or a platform-neutral shared implementation, as I 
said originally.  You didn't seem to like those options.  Yet you never 
proposed a viable alternative.

> In other words: strict modularity.

NSPR is quite modular in this sense, yes?

> If you need new semantics: build a new, dedicated library and *import*
> (NOT include) it into your main project.

Yes.  See NSPR.

> Platform dependend exceptions (workarounds) are the worst approach:
> For an large application, it makes it unmaintainable, and all that
> work is very redundant.

Hey, we agree on something!

> Reimplementing some functionality (eg. of libc) is unproblematic, 
> as long as it's done in an dedicated library, which doesn't conflict
> existing ones and is open to the world.

I'm failing to see what your problem with the existing setup is, then.

> For a *short* time, it's okay to work around other's bugs. But this has
> to be *strictly limited*

Define "short"?  It's normal for bugs to take years to fix, if they get 
fixed at all.

Complexity has to be managed, yes.  News at 11.

> The long release cycles of certain packages (eg. glibc) have their 
> reasons: the same we're currently talking about. Simply too large!

The problem is not just the libc cycle but also the OS vendor release 
cycle...  And then the fact that most people don't upgrade their OS the 
instant an upgrade is available.

> IMHO it's better to help solving the actual problem instead of trying
> to work around it (and so introduce the risk of new bugs).

Yes, that's great when it can happen and is the obvious best course of 
action.  It's not always available.

> Then it might be broken. The OS vendor is responsible for providing 
> an working OS - no one else.

That doesn't solve the pernicious problem of trying to build software 
that will run on said OS (all of them, to be exact).

> I don't see any reason why that new allocator should be limited to
> the only one client package.

Who ever argued that it would be?

> Some time ago, I already asked for the reasons. For those few cases
> I got an answer, it were just people not wanting to wait for some
> bugs get fixed.

For severe enough bugs (security bugs, e.g.) this might be a reasonable 
tradeoff.

> Ah, so you also feel responsible for certain people's broken our 
> outdated systems ? Oh, that's interesting.

When "certain people" is "100% of our userbase"... yes, maybe we feel we 
need to deal with the situation somehow.

> Maybe you also feel responsible for my dirty dishes and like to 
> come around for cleaning the up ? ;-P

Perhaps not, but I see no problem with selling food that won't stick to 
your dishes if I know that all my customers have no running water to 
clean dishes with.

> To the comments you're refering to.

You mean the start of this thread?  The original mail you replied to?

I'd assumed you read it before replying...

-Boris