Re: PERT fit with curve_fit function

Hello,

Thank you for your answer,

Do you have an example, please ?



Le mar. 20 déc. 2022 à 09:27, Kevin Sheppard <[email protected]> a
écrit :

> If you want a fast method, you should consider a method of moments
> estimator.  You could use the mean, variance, and skewness of your data to
> exactly identify the parameters of the model.  These would lead to
> consistent estimates assuming that the model was correctly specified,
> although it will likely differ from maximum likelihood estimates when it is
> not.
>
> Kevin
>
>
> On Tue, Dec 20, 2022 at 8:20 AM Julian CHAMBRIER <
> [email protected]> wrote:
>
>> Hello Robert,
>>
>> Thank you for your answer,
>>
>> As documented, if you don't provide an initial guess `p0`, it will use
>>> `np.ones()` to make the initial parameter vector. This happens to be a
>>> degenerate input for you since `low==peak==high`, so it will likely not see
>>> an improvement by changing the parameters in its initial steps, so it will
>>> think that it has already converged or won't converge. You can try
>>> providing a more appropriate `p0`, say `[x.min(), (6 * x.mean() - x.min() -
>>> x.max()) / 4, x.max(), 4.0]`.
>>>
>>
>> > Yes indeed it's a bit better but the results differ a lot, which I
>> suspected.
>>
>>
>> 2. If you must fit it, fit the Beta distribution using the existing
>>> algorithms and convert the fitted parameters to the PERT parameterization.
>>>
>>
>>
>> >  Following your advice, I use the beta distribution where I modify the
>> parameters to match a Pert distribution.
>>
>> Nevertheless, I have a particular need. Indeed, in many cases the beta
>> inference is enough, but on samples that do not necessarily follow a Pert
>> distribution, the beta works very badly (which is normal).
>>
>> I would have liked a fast alternative that would infer (roughly) a Pert
>> on any sample, in case the Beta inference does not give me good results.
>> You would have understood that my need is to represent any data sample with
>> a Pert.
>>
>> Granted, sometimes other distributions will be much more suitable, but
>> that will be in a second time.
>>
>> That's why I'm looking at other alternatives to the Beta.
>>
>> To summarize:
>> 1) I infer with a Beta in case the inference goes well and the sample is
>> representative of that distribution. I then change the parameters to a PERT
>> distribution.
>> 2) If the Beta inference is not conclusive, I must have an alternative
>> that finds me the Pert distribution that best represents my data (even if
>> the data do not follow this distribution). I would have liked an
>> alternative that is rather fast in time, but that is more efficient than
>> just retrieving a min, a max, ...
>>
>> By the way, I have another question. To know if the beta distribution
>> goes well I test if a>1 and b>1. But I realize that in some cases, a and b
>> are quite large and therefore the scale parameter (of the Pert
>> distribution) is also very large. Is there any other case where we can have
>> an intuition that the inference with Beta has failed (maybe on the scale or
>> loc parameters).
>>
>> Thanks in advance,
>>
>> Julian
>>
>>
>>
>>
>>
>> Le lun. 19 déc. 2022 à 22:06, Robert Kern <[email protected]> a
>> écrit :
>>
>>> On Mon, Dec 19, 2022 at 3:30 PM Julian CHAMBRIER <
>>> [email protected]> wrote:
>>>
>>>> Hello,
>>>>
>>>> I tried to use the curve_fit function to find the low, peak, high and
>>>> lmb parameters that minimize the error of the probability function of the
>>>> PERT distribution, but I'm having some trouble.
>>>>
>>>> I can't see my error, and I keep getting [1. 1. 1. 1.] outputs for the
>>>> parameters.
>>>>
>>>> Do you have any idea?
>>>>
>>>
>>> As documented, if you don't provide an initial guess `p0`, it will use
>>> `np.ones()` to make the initial parameter vector. This happens to be a
>>> degenerate input for you since `low==peak==high`, so it will likely not see
>>> an improvement by changing the parameters in its initial steps, so it will
>>> think that it has already converged or won't converge. You can try
>>> providing a more appropriate `p0`, say `[x.min(), (6 * x.mean() - x.min() -
>>> x.max()) / 4, x.max(), 4.0]`.
>>>
>>> I do not recommend using `curve_fit()` to fit a PDF against histogrammed
>>> data. It's basically always worse theoretically and practically than doing
>>> a maximum likelihood estimate. You can see my arguments (with links) in
>>> this issue:
>>>
>>>   https://github.com/numpy/numpy/issues/13194
>>>
>>> I reiterate my advice:
>>>
>>> 1. Don't try to fit a PERT distribution to data. That's not why it
>>> exists. It's not flexible enough to fit the kind of data that you want to
>>> throw at it, so you will generally get nonsensical results from any fitting
>>> procedure. You are encountering an example of the Folk Theorem of
>>> Statistical Computing
>>> <https://statmodeling.stat.columbia.edu/2008/05/13/the_folk_theore/>:
>>> When you have computational problems, often there’s a problem with your
>>> model.
>>>
>>> 2. If you must fit it, fit the Beta distribution using the existing
>>> algorithms and convert the fitted parameters to the PERT parameterization.
>>>
>>> --
>>> Robert Kern
>>> _______________________________________________
>>> SciPy-Dev mailing list -- [email protected]
>>> To unsubscribe send an email to [email protected]
>>> https://mail.python.org/mailman3/lists/scipy-dev.python.org/
>>> Member address: [email protected]
>>>
>> _______________________________________________
>> SciPy-Dev mailing list -- [email protected]
>> To unsubscribe send an email to [email protected]
>> https://mail.python.org/mailman3/lists/scipy-dev.python.org/
>> Member address: [email protected]
>>
> _______________________________________________
> SciPy-Dev mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> https://mail.python.org/mailman3/lists/scipy-dev.python.org/
> Member address: [email protected]
>

_______________________________________________
SciPy-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/scipy-dev.python.org/
Member address: [email protected]