Re: Percentiles (still hurting my brain)

On Sunday 28 March 2004 20:38, John Darrington wrote:
> You wouldn't belive such a simple concept could cause so much trouble!

Seems like a lot of self-abuse for an ill-defined statistic!

> I've implemented Michael's algorithm in frequencies.q whcih we can run
> with if we so desire.  If however, we want the SPSS one, then I'm all
> at sea.

But with the indexing in my second e-mail on the subject:
double idx = (n+1)*p - 1; // for zero-based arrays

> Let's run the data (1 2 3 4 5)  through  it, searching for the 50th
> percentile:
>      _SPSS Statistical Algorithms_ says, roughly:
>
> W in this case is 5.
>      The p'th percentile:
>
>      Find the first score interval (x2) containing more than tp cases.
>
> Well further down it says that tp=(W+1)*p/W so tp=6*50/5 = 60
> So the first problem is that there is no score with more than 60
> cases.  So I assume that either Ben, or the typsetter of the book made
> a mistake, and should have typed tp=(W+1)*p/100 in which case
> tp = 6*50/100 = 3.
> Thus x2 = 4 and x1 = 3
> Now cc1 = 1+2+3 / 1+2+3+4+5 = 10/15 = 2/3
> and therefore cp1 = 200/3

I think cc1 should be 3 according to the definition that Ben gave.  cp1 sould 
be 3/5.

> so tp - cp1 = 3 - 200/3  which doesn't satisfy the condition
> tp-cp1 >= 100/W

I'm not really sure what this is supposed to do.  I suspect it's trying to 
trap trivial cases, but the inequation is way off.  Maybe it's supposed to be 
tp-cc1 > 0   

> So that answer is the 2nd case, viz:
>
> 1 - ((W+1)p/100 - cc1) x1 + ((W+1)p/100 - cc1)x2
>
> which can be simplified to
...

That's probably the first clue that there's a serious typo.  Granted, it 
probably also would have been a great place for the original author to 
substitute tp instead of (W+1)p/100.  Oh well.

My algorithm (with indexing in second e-mail) returns the same results 
reported by SPSS as long as all the weights are equal.

So my guess is if tp-cp1 > 0 then (for 1-based arrays):

(1 - (tp-cc1))x1 + (tp-cc1)x2

This is just a rearrangement of what I wrote to make it look more like the 
weighted mean of two values.  I guess they misplaced a parenthesis.

We should put that in the documentation, too.

How does that work?

- M.