Scipy Function for T-Test on Lognormal distributions

edo-cnl · April 18, 2025, 3:46pm

Is your feature request related to a problem? Please describe.

In Scipy, there is no function to conduct T Testing on lognormal distributions, even though there are approaches that seemingly provide good coverage of the Confidence Interval.

These approaches, namely the Modified Cox Method, are available in R (R: Estimate Parameters of a Lognormal Distribution (Original...) which to me hints to an existing gap in Scipy.

Describe the solution you’d like.

I found myself in the position of having to research and craft the function for the the T Test lognormal distribution myself.
I believe it would enrich Scipy as well as all its users if a function, with all the required parameters, was available.

The function I wrote is similar to one_tail_test_lognormal_distribution_modified_cox.

Describe alternatives you’ve considered.

In Olsson 2005 there are different methods used to construct the approximate confidence intervals.
The “Modified Cox Method” seems to be one that performs the best in achieving the desired coverage of the CI, particularly with small sample sizes.

Additional context (e.g. screenshots, GIFs)

I replicated the experiment in a notebook, and the results were similar to the experiment in Olsson’s paper.
This is the second issue I open on this topic. Previously opened issue.

josef-pkt · April 22, 2025, 5:17pm

I have open issues for something like this in statsmodels, but I looked only at the regression cases.
e.g.

github.com/statsmodels/statsmodels

ENH: tools bias correction for nonlinear transformation

opened 03:46PM - 27 Mar 20 UTC

josef-pkt

type-enh comp-stats comp-tools

there are several issues related to this #3224 transformation to normality #27…91 backtransform for transformed endog, e.g. log(endog) in time series, smearing also related delta method This issue is specifically for collecting nonlinear transformations with helper methods to estimate the bias correction of nonlinear transformation. standard case: log-normal <-> normal current usecase I would like to get a quick idea about bias when we convert between several measures e.g. effect size for anova, https://github.com/statsmodels/statsmodels/pull/6526/files#diff-26abd2f0b72810fe723fb299c04911e9R123 (I don't want to bother to have to derive the bias by hand for each case, which is mainly getting the derivatives)

github.com/statsmodels/statsmodels

ENH: predict, get_prediction with log or transformed endog

opened 05:53AM - 31 Jan 16 UTC

josef-pkt

type-enh comp-base comp-regression prio-elev

How do we get prediction and similar for models with transformed endog? example …`log(y) ~ ...` estimated with OLS related to issue #1449 #1480 also related: use Poisson instead, several articles and FAQ also GLM-Gamma is close to lognormal model specific point: correcting the prediction of the original so we get E(y | x) or E(y | ?) for population. We have an explicit expression for the correction under lognormality assumption. in general we can use a _smearing_ estimate (I went through this some time ago, but don't know where the functions are. I don't see a specific issue. I just bumped into this blog again.) see for example http://davegiles.blogspot.ca/2014/12/s.html In equation (3) he uses the average exp residuals. AFAIR, I averaged over sample x_i, which might (!) be somewhere in Wooldridge. It's not clear yet how to connect this to the models (especially we don't know which transformation the user has used), but we can provide at least some helper function. It fits most likely in the topic of various predict functions and methods. (PR #2296 )

(and several other open issues)

However, I have no overview which methods would be worth including in statsmodels (and we still don’t have a log-normal regression model directly)

For the “stats” case, hypothesis tests and confidence intervals for one and two sample cases (which I never looked at for log-normal), it would be easier to provide different methods with better small sample properties than just asymptotics, similar to what we have for proportion and rates (binomial and poisson).
.