EM-type algorithms for computing restricted MLEs in multivariate normal distributions and multivariate t-distributions

Constrained parameter problems arise in a variety of statistical applications but they have been most resistant to solution. This paper proposes methodology for estimating restricted parameters in multivariate normal distributions with known or unknown covariance matrix. The proposed method thus provides a solution to an open problem to find penalized estimation for linear inverse problem with positivity restrictions [Vardi, Y., Lee, D. 1993. From image deblurring to optimal investments: Maximum likelihood solutions for positive linear inverse problems (with discussion). Journal of the Royal Statistical Society, Series B 55,569-612]. By first considering the simplest bound constraints and then generalizing them to linear inequality constraints, we propose a unified EM-type algorithm for estimating constrained parameters via data augmentation. The key idea is to introduce a sequence of latent variables such that the complete-data model belongs to the exponential family, hence, resulting in a simple E-step and an explicit M-step. Furthermore, we extend restricted multivariate normal distribution to multivariate t-distribution with constrained parameters to obtain robust estimation. With the proposed algorithms, standard errors can be calculated by bootstrapping. The proposed method is appealing for its simplicity and ease of implementation and its applicability to a wide class of parameter restrictions. Three real data sets are analyzed to illustrate different aspects of the proposed methods. Finally, the proposed algorithm is applied to linear inverse problems with possible negativity restrictions and is evaluated numerically.