__malloc_init_space (more examples)

> Unfortunately, in the generic case, the static buffer trick is
> actually detrimental, because it's impossible to reclaim the 4 kB once
> they're used.

Laurent Bercot write above yesterday.  Sorry, Laurent, that is not
_true_ (for the malloc I suggested)!

Actually malloc uses mmap/munmap for size >= PAGE_SIZE.
For small pages it mmap one page and shares it for small sizes.
The worst case is to have 8 open pages and each have
16, 32, ..., 2048 unused bytes.

More interesting is what happen by free(x).  If the page is not small
it is unmapped.  If the page is small it is zeroed and then put in a 
stack.  Then it can be used again.  We cannot "true" free next:

	int k;
	char *x[256];
	for (k=0; k<256;) x[k] = malloc[900];
	for (k=0; k<256;) free(x[k]);

Now how to use __minit_init_space?  (if this is a global pointer;
see my dietfeatures.h)

 typedef struct __alloc_t { void *next; size_t size; } __alloc_t;
 extern __alloc_t *__malloc_init_space;
 static __alloc_t x_space[(8*4096)/sizeof(__alloc_t)];

 int main(int argc, char **argv) {
 ...
 x_space->size = sizeof(x_space);
 x_space->next = __malloc_init_space;
 __malloc_init_space = x_space;

This means the following:  we help to malloc and say him
to use x_space for small pages.  It is the same as mmap.
free() work again.  By DJB, it is impossigle to use the static 
buffer again after free.  If we have fresh new_space (may be zeroed)
we can use above many times in our program like:

 new_space->size = sizeof(new_space);
 new_space->next = __malloc_init_space;
 __malloc_init_space = new_space;

Each time the new_space must be different!  The only restriction are:
  1) new_space must be sizeof(__alloc_t) alligned
  2) sizeof(new_space) must be multiple of 2*sizeof(__alloc_t)

Since static buffers are in BSS they are zeroed.  I have tried also:
int main() {
    __malloc_init_space(alloca(8192));
    __malloc_init_space->size = 8192;
    __malloc_init_space->next = 0;

    /* no guarantee that all is zeroed by alloca !!! */
    ...
}

May be the most interesting case is to see what happen in memory 
for small pages.  We need to include in program the following:

void *small_init(char *name, unsigned int len) {
  extern __alloc_t *__malloc_init_space;
  __alloc_t *t;
  int fd;

  unlink(name);
  fd = open(name, O_EXCL | O_RDWR | O_CREAT, 0600);
  if (fd<=0) return 0;
  ftruncate(fd, len);
  t = mmap(0,len,PROT_READ|PROT_WRITE,MAP_SHARED,fd,0);
  if (t==(__alloc_t *)-1) return 0;
  close(fd);
  
  t->size = len;
  t->next = __malloc_init_space;
  __malloc_init_space = t;
  return t;
}

int main() {
  some declarations;
  small_init("/tmp/__#malloc.txt", 512*1024);
  rest of code;
}

I have tried above for different programs.  For example ssh
uses about 90-100K for small pages.  In my package there is 
a tst-pthread program.  It start 52 threads and uses one such
512K file for all threads.  Work good.  See with an editor 
what happen in "/tmp/__#malloc.txt".

Resume:  If you find the pointer __malloc_init_space a big security
hole make it _static_, please!

Regards,  Nikola

--
PS.  In my systems I see that alloca and static buffers are in BSS
section always.  The program uses only the _needed_ space.  If we 
don't touch it it remains in the kernel!  Is this always true?

I have tried to munmap also small pages.  This is lots of work
for nothing.  The effect is too small.  The example above
	for (k=0; k<256;) x[k] = malloc[900];
is very exotic!  I have a program which make statistic.  If somebody
is intrested -- welcome!

http://riemann.fmi.uni-sofia.bg/programs/diet/djb_alloc.tar.gz
http://riemann.fmi.uni-sofia.bg/programs/diet/alloc.tar.gz