As a flexible and powerful statistical tool, finite mixture models, in particular Gaussian mixture models (GMMs) have been widely studied and used in the various domains such as pattern recognition, data mining, image analysis due to their computational tractability, ease to implement, and capability of representing arbitrarily complex probability density function with high accuracy. One common approach to parameter estimation of mixture models is to iteratively calculate maximum likelihood (ML) solution via expectation-maximization (EM) algorithm. However, the ML estimate calculated by conventional EM algorithm for GMMs is sensitive to atypical samples, which usually exist in the practical application due to the factors such as background noise or sampling errors. In addition, the number of the components of mixture models (model order) is assumed to be known beforehand. Otherwise, the estimation result may be rather poor. This thesis presents a novel method based on deterministic annealing (DA) framework to handle the problems of the sensitivity to atypical observations and initial configuration of number of components associated with the ML estimate. Under the DA framework, the parameters of mixture models are learned via trimmed likelihood estimate (TLE) and Bayesian information criterion (BIC). In addition, since the learning of parameters, model selection and outlier detection are simultaneously performed, and the learning process is controlled by a sequence of gradually lowered temperatures based on the principle of maximum entropy, the local optima problem can also be alleviated.