#159 wrong use of exact distribution with robust estandard errors

[Pedro AP]

It seems that current results when using robust standard errors display p-values computed from exact Student-t distributions. This is wrong since the exact distribution of OLS is unknown under heteroskedastity (or serial correlation). Then one can only use that t-ratios follow an asymptotic standard normal distribution. It would be also probably better to use the Wald statistic (instead of the "robust" F statistc) when testing linear restrictions.

[Good_Newz]

I compared the output from getl and Stata with the robustness command. The p-values are identical. I think gretl (and Stata) adjust the covariance matrix so that t-values from a normal distribution can be used.

[Pedro AP]

Dear good_newz,
I also thought that t-ratios should be corrected as you mentioned, but I am sorry to tell you that, at least in my version of gretl this is not the case; I have checked this in the latest stable for Windows (just downloaded last Saturday 9th July) and in version 1.9.3 available for Ubuntu in repos. I don't know what Stata or other programs do, and actually you do not need to go outside gretl to check the wrong use of p-values I am reporting.

If I just open the sample dataset "banks91" and run a simple regresion of Y on a constant a P1 with robust standard errot, I get a t-ratio for the constant of -3.194 and a reported p-value of 0.0016. Then you can simple use the tool for cumputing p-values available in the very same gretl to see that
(1) the two-sided value of -3.194 for a standard normal is 0.00140316 (which cannot be rounded to 0.0016)
(2) the two-sided value of -3.194 for a Student-t with 196 degrees of freedom is 0.00163512 (from which the reported p-value is likely to come from)

So, I am afraid to say that independently of what we all know that gretl (and others) should do, it seems that gretl does not. Maybe it is different with your data or with your version of gretl....

[Good_Newz]

Dear Albarran,

I now understand your comment. Indeed the p-values do come from t-distribution. I know you say the error distribution is of unknown type. Why would you then use a normal distribution to get the p-values (remember you dont know the distribution type)?

[Allin Cottrell]

I agree with good_newz.

There's an asymptotic argument that "t-statistics" constructed using
robust standard errors will tend to follow the normal distribution as
the sample size tends to infinity, but

(a) If you have a really big sample, it makes no appreciable
difference whether you use the z or t distribution, and

(b) In a small to moderate-sized simple use of the Student t
distribution is more conservative (in the sense of not over-stating
the degree of evidence against the null), and is also standard.
Stata certainly does the same as gretl, and I strongly suspect
most econometric software does.

[Pedro AP]

Dear allin (and good_newz),

I am sorry but I strongly disagree with you. Your arguments might be "vaguely" and "approximately" true, but are conceptually wrong. This is important because econometric and stastistic resutls are supposed to have credibility because of its well-founded mathematical concepts. Your arguments are often given by occasional practitioners (they are kind of arguments from "recipe-book" of "applied" econometrics) but true applied econometricians (not to speak of theoretical econometrician) would say that are invalid. You will not find those arguments in any serious econometric textbook . Actually my graduate student will fail if they argue in that way :) .

In the linear regression model (in which tha unconditional variance of the error term is also estimated), the OLS estimators and therefore the "t-ratios" follow a Student-t distribution ONLY IF the errors follows a normal distribution and under homoskedasticity and lack of serial correlation. If any of these conditions are not met, then the distribtuion of the OLS estimators is no longer a Student-t distribution; so you cannot argue in any way that is conceptually ok to use them. However one can still use the result that the asymptotic distribution of OLS is always a normal one.

So it is clearly wrong to use Student-t in a context in which is not valid. It has nothing to do with being more or less conservative in small or large sample: p-values from Student-t are invalid.

Moreover, your argument about "asymptotic" is at least inexact. what the theory says is that Student-t converges to the normal. But in this case, you do not any statistic following a Student-t. Moreover you "propose" using Student-t as an "approximation" to the normal in large samples! This is a surprising change in the asymptotic theory: I have never seen an econometrics or statistics book that assert that an asymptotic distributions is Student-t.Why using an approximation (Student-t) to the well-known normala distribution? Asymptotic distributions is useful because you can use normal distribution instead of (as an approximation to) any unknown/not clearly specified distribution. Why approximating the approximate/asymptotic distribution? Again some practitioners woull say the same as you: it does not make a difference in practice, but it is not that important, why using something conceptually wrong instead of the similar wright thing?

All this is independent of what other econometric packages do. If they do the same wrogn thing, shame on them!. It is really simple to include an if-stament to use the normal distributions if the option of robust standard errors is checked. Actually one should be given the option in general (as I have asked for in another thread) to use asymptotic tests since I do not want to believe that, despite of ignoring many things about the model, I know for sure the distribution of the error term.

Best regards,
Pedro

[Sven S.]

[This action is part of a little round of maintenance, sorry for the delay.]

Well, as Allin said, there is no difference between a t(n)-distribution and a standard normal for $n \rightarrow \infty$. This is an exact mathematical statement in an appropriate asymptotic limiting sense, not just "vaguely" or "approximately" true. Now you can make the case that if only asymptotic results are known then any finite-sample analysis should be forbidden, but that is surely an extreme position and also not very helpful. Otherwise it's all a question about which statistics provide the best approximations to the (generally unknown) exact finite-sample distributions. Opinions differ and depend on context, but following the standards of other packages is surely not a bug. So I'm closing this.

thanks,
sven

[Pedro AP]

There is a the very least a strong inconcruency in Gretl (and others): the same discussion above applies to Instrumental Variables estimators where only asymptotic distribution is known (normal for "t-ratios"). Normal distribution is, of course, used for p-values and even rename t-ratios to z-ratio to emphasize this. Why not using t-distributions if it is in practice equivalent to normal and conceptually not invalid?
The point is that it is actually conceptually wrong (not an extreme position, as in rigorous science, things are right or not, beyond what the majority thinks) to claim that we know or we can approximate the exact distribution of IV (and OLS with Robust SE), so only claims about the known asymptotic distribution are done; and this is normal (Central Limit Theorem), althogh t-distribution converges to normal as $n\to\infty$.

I felt that Gretl had the opportunity to do the things right, to make things clear and to be congruent (using the same criteria in two equivalent cases) beyond what others do and what in practice "just works". Maybe I was wrong in this point.