Re: libcruft/sysconf_cpus.c patch

Hello,

On Wed, November 24, 2010 14:19, Bela Lubkin wrote:
> Indan Zupancic wrote:
>
>> So I'd argue that dietlibc is free to do what it wants here. Everyone using
>> this interface wants to know how many cpus they can actually use anyway.
>
> If they want to know how many they're currently setup for, they can use
> getaffinity.  The point of having a getconf call is to get below that to
> what the OS has available.
>
> Then if the process has enough privilege (which is no special priv on
> Linux, by default), it can use setaffinity to turn back on processors it
> was started without.  Even if it has insufficient privilege, it can
> social engineer: "dear user, this will perform better if you run it with
> all CPUs enabled".

You don't need any special privileges for setaffinity. The process just has
to be either root, or the same user.

>> This you can find out with with one sched_getaffinity() call. Then there's
>> the theoretical case of more cpus becoming available after the call, but as
>> this is very unlikely and so obscure that I don't think dietlibc should
>> bother getting this corner case right.
>
> This is the wrong year to be calling it obscure: virtualization
> platforms are busily adding hot-plug VCPU support.

There is no way, as far as I know, besides udev, to get notifications of
more cpus coming online. And if you're concerned about the actual number
of cpus then you don want to know any theoretical number, just the real one.

That said, knowing the maximum possible number of cpus could be useful
sometimes. But not generally enough to bother with in dietlibc I think,
until some real user actually requests it.

>> > There are at least 3 numbers one *could* want to know: (1) # CPUs in the
>> > hardware, (2) # CPUs activated by the system (e.g. if the OS has CPU
>> > licenses, or detected some are broken), and (3) # CPUs *this* process is
>> > allowed to access.  The doc says (to me) #2.
>>
>> Knowing 1 gives you nothing useful, neither does 2 if 3 is different.
>
> #2 is like "ulimit -H #cpus" and #3 is "ulimit -S #cpus", it is worth
> while knowing the difference.

Yeah, sometimes, I suppose.

>> > #if !defined(SLASH_PROC_OK) || defined(__arm__)
>> > int __sc_nr_cpus() {
>> >   return 1;	/* kludge kludge ;-) */
>> > }
>>
>> Some new ARMs have SMP support.
>
> Ok.  That was just carried in from old code.
>
>> Shouldn't you check if nr is greater than 0, to catch untested platforms?
>
> Probably.  The old code didn't.
>
>> For comparison, the same using sched_getaffinity() would look something
>> like this:
>>
>> #include <sched.h>
>>
>> #define NR 8
>>
>> int __sc_nr_cpus(void)
>> {
>> 	int i, c;
>> 	unsigned long v;
>> 	unsigned long m[NR];
>>
>> 	if (sched_getaffinity(0, sizeof(m), &m))
>> 		return 1;
>> 	for (c = 0, i = 0; i < NR; i++){
>> 		/* Peter Wegner/Derrick Lehmer/Brian Kernighan's method */
>> 		for (v = m[i]; v; c++){
>> 		  v &= v - 1; // clear the least significant bit set
>> 		}
>> 	}
>> 	return c;
>> }
>
> Pleasingly simple, at least.
>
> Looking upwards at sysconf.c (caller of __sc_nr_cpus()), I see there's
> little attempt to read out the real system at hand.  So I see I'm
> inappropriately arguing for accuracy in a setting of general laxness.
> Never mind.

Yeah, as far as I know programs just use it to guess how many threads to
use to perform well. And for that using getaffinity seems fine.

I'll try to find some time writing adding sched_[gs]etaffinity syscall
and the above using it. (Except if someone beats me to it.)

Greetings,

Indan