Re: PERT fit with curve_fit function

Hello Robert,

Thank you for your answer,

As documented, if you don't provide an initial guess `p0`, it will use
> `np.ones()` to make the initial parameter vector. This happens to be a
> degenerate input for you since `low==peak==high`, so it will likely not see
> an improvement by changing the parameters in its initial steps, so it will
> think that it has already converged or won't converge. You can try
> providing a more appropriate `p0`, say `[x.min(), (6 * x.mean() - x.min() -
> x.max()) / 4, x.max(), 4.0]`.
>

> Yes indeed it's a bit better but the results differ a lot, which I
suspected.


2. If you must fit it, fit the Beta distribution using the existing
> algorithms and convert the fitted parameters to the PERT parameterization.
>


>  Following your advice, I use the beta distribution where I modify the
parameters to match a Pert distribution.

Nevertheless, I have a particular need. Indeed, in many cases the beta
inference is enough, but on samples that do not necessarily follow a Pert
distribution, the beta works very badly (which is normal).

I would have liked a fast alternative that would infer (roughly) a Pert on
any sample, in case the Beta inference does not give me good results. You
would have understood that my need is to represent any data sample with a
Pert.

Granted, sometimes other distributions will be much more suitable, but that
will be in a second time.

That's why I'm looking at other alternatives to the Beta.

To summarize:
1) I infer with a Beta in case the inference goes well and the sample is
representative of that distribution. I then change the parameters to a PERT
distribution.
2) If the Beta inference is not conclusive, I must have an alternative that
finds me the Pert distribution that best represents my data (even if the
data do not follow this distribution). I would have liked an alternative
that is rather fast in time, but that is more efficient than just
retrieving a min, a max, ...

By the way, I have another question. To know if the beta distribution goes
well I test if a>1 and b>1. But I realize that in some cases, a and b are
quite large and therefore the scale parameter (of the Pert distribution) is
also very large. Is there any other case where we can have an intuition
that the inference with Beta has failed (maybe on the scale or loc
parameters).

Thanks in advance,

Julian





Le lun. 19 déc. 2022 à 22:06, Robert Kern <[email protected]> a écrit :

> On Mon, Dec 19, 2022 at 3:30 PM Julian CHAMBRIER <
> [email protected]> wrote:
>
>> Hello,
>>
>> I tried to use the curve_fit function to find the low, peak, high and lmb
>> parameters that minimize the error of the probability function of the PERT
>> distribution, but I'm having some trouble.
>>
>> I can't see my error, and I keep getting [1. 1. 1. 1.] outputs for the
>> parameters.
>>
>> Do you have any idea?
>>
>
> As documented, if you don't provide an initial guess `p0`, it will use
> `np.ones()` to make the initial parameter vector. This happens to be a
> degenerate input for you since `low==peak==high`, so it will likely not see
> an improvement by changing the parameters in its initial steps, so it will
> think that it has already converged or won't converge. You can try
> providing a more appropriate `p0`, say `[x.min(), (6 * x.mean() - x.min() -
> x.max()) / 4, x.max(), 4.0]`.
>
> I do not recommend using `curve_fit()` to fit a PDF against histogrammed
> data. It's basically always worse theoretically and practically than doing
> a maximum likelihood estimate. You can see my arguments (with links) in
> this issue:
>
>   https://github.com/numpy/numpy/issues/13194
>
> I reiterate my advice:
>
> 1. Don't try to fit a PERT distribution to data. That's not why it exists.
> It's not flexible enough to fit the kind of data that you want to throw at
> it, so you will generally get nonsensical results from any fitting
> procedure. You are encountering an example of the Folk Theorem of
> Statistical Computing
> <https://statmodeling.stat.columbia.edu/2008/05/13/the_folk_theore/>:
> When you have computational problems, often there’s a problem with your
> model.
>
> 2. If you must fit it, fit the Beta distribution using the existing
> algorithms and convert the fitted parameters to the PERT parameterization.
>
> --
> Robert Kern
> _______________________________________________
> SciPy-Dev mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> https://mail.python.org/mailman3/lists/scipy-dev.python.org/
> Member address: [email protected]
>

_______________________________________________
SciPy-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/scipy-dev.python.org/
Member address: [email protected]