Re: Support for complex data in stats.pearsonr

What if we made these test methods (permutation_test, monte_carlo_test,
bootstrap) classes with a __call__ method which could be called
recursively from within the statistic.

>From a user's point of view, it would look something like this:

mytest = permutation_test(my fav. arguments)
rho, pvalue = pearsonr(data, significance=mytest, *)

This lets us encapsulate parameters particular to the tests to the test
class rather than having to deal with them within the statistic's
method.  We can check test parameters from within the statistic methods
to prompt warning messages when test parameters may not make sense.  We
may also want to introduce a "None" option to the significance argument
so that some computation time is saved in the test.  In the case of the
current default pvalue/confidence estimation some string argument to the
significance parameter (e.g. significance="default" or "parametric") 
would turn on that behavior.  

This would, I think, add a lot of functionality with a minimal change to
the API.  Setting up a particular kind of test would be similar to how
one uses the statistical distribution classes.

Perhaps with that thought, rather than a __call__ method, we could
implement a test method, a CI method, etc.  The call pattern would mimic
the statistical distribution API.

- Chris

On Thu, Feb 16, 2023 at 03:15:41PM -0800, Matt Haberland wrote:
> After the addition of `stats.permutation_test`, `stats.monte_carlo_test`,
> and `stats.bootstrap`, the natural next step was to make them easier to use
> with statistics that SciPy already defines. IMO, the main question has been
> API.
> 
> I added a permutation version of `anderson_ksamp` in [3] to resolve [4],
> but I wanted to decide on a more longer-term plan before adding that sort
> of feature to many other functions.
> 
> Some questions to consider:
> 
>    - If we want a function like `pearsonr` to perform a permutation test,
>    how should the user specify the required information?
>       - Parameters `n_resamples` and `random_state` as we did in [3]?
>       - A single object with both of these pieces of information (and
>       possibly more configuration information) to avoid adding several new
>       parameters to the function?
>       - Alternatively, should the user just call `permutation_test` with
>       the callable as `stats.pearsonr`, and we can automatically choose some
>       appropriate settings for the test (e.g. the permutation type)?
> 
> 
>    - In many cases, it can also make sense to perform a Monte Carlo test
>    using the same statistic.
>       - How does the user specify whether to perform the Monte Carlo
>       version of the test or the permutation version (e.g. a `method`
> parameter)?
>       - There is often a natural choice of distribution(s) from which to
>       draw Monte Carlo samples (e.g. for a two-sample t-test, draw samples from
>       the standard normal). But if one knows something about the
> distribution(s)
>       from which their data is drawn, it might make sense to customize the test
>       using that information but still use the t-statistic. Should there be
>       options for that, or at that point, do we just direct the user to use
>       `monte_carlo_test` directly?
>    - We can ask the same sorts of questions about the `confidence_interval`
>    method of `pearsonr` and other functions. For some tests, one can envision
>    getting confidence intervals not only via the bootstrap, but also using
>    permutation or Monte Carlo ideas!
> 
> I'd be happy to discuss this further. I've been interested in this for a
> while; I've just put this on the back burner because there are a lot of
> questions to answer ahead of time (unless we are OK with every function
> re-inventing the interface).
> 
> Please feel free to open an issue!
> 
> Matt
> 
> [3] https://github.com/scipy/scipy/pull/16996
> [4] https://github.com/scipy/scipy/issues/9527
> 
> 
> On Thu, Feb 16, 2023, 2:01 PM Christopher Cowden <[email protected]>
> wrote:
> 
> > Hi,
> >
> > A recent issue [1] brought up the idea of adding support for complex data
> > in stats.pearsonr.  It seems like this would be a useful feature,
> > especially for folks working in signal processing.  It wouldn't be too
> > difficult to implement, see [2].  The current estimation of p-values,
> > however, seems to underestimate the significance.  That is, the error rate
> > tends to be lower than the p-value.  It would be possible to introduce a
> > permutation test to estimate the p-value of the correlation between complex
> > numbers.
> >
> > Is this something worth pursuing?  What do you think of introducing an
> > alternative p-value estimation method?
> >
> > Cheers,
> > Chris
> >
> > [1] https://github.com/scipy/scipy/issues/17518
> > [2] https://en.wikipedia.org/wiki/Complex_random_variable
> > _______________________________________________
> > SciPy-Dev mailing list -- [email protected]
> > To unsubscribe send an email to [email protected]
> > https://mail.python.org/mailman3/lists/scipy-dev.python.org/
> > Member address: [email protected]
> >

> _______________________________________________
> SciPy-Dev mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> https://mail.python.org/mailman3/lists/scipy-dev.python.org/
> Member address: [email protected]

_______________________________________________
SciPy-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/scipy-dev.python.org/
Member address: [email protected]