This web-page is a beginning attempt to review the applications of finite mixture modeling. I expect that it will be far from complete but hopefully it will grow and be more helpful to individuals as I have a chance to revise it from time to time.
The idea behind mixture modeling is pretty easy to understand. It suggests that a distribution that may appear normal, skewed, bi-modal, etc., may in reality be made up of more than one underlying distributions which combine to give it that shape. For example, a type of bird called Water Pipits (Anthus spinoletta) tend to have wing lengths that are either about 86 millimeters or 92 millimeters long depending on their gender (Flury, Airoldi, & Biber, 1992). Similarly, the length of halibut fish of a particular age show different distributions for males and females (Hosmer, 1973). Where it is difficult to determine the gender of a halibut or a water pipit, knowing its size could provide a pretty good idea of the underlying gender. Mixture modeling provides a statistical way of taking a single distribution and breaking it up into subgroups to better describe the reality that is being observed. The word finite is often included referring to the fact that there is a limited number of groups which are hypothesized to underlie a distribution. Most applications are limited to two or three groups.
In the case of a skewed distribution, it may be that there is an underlying skewed distribution (e.g. more individuals tend to be optimistic than pessimistic), or it may be that there are more than one distributions which are overlapping to give the impression of a skewed distribution (e.g. in filling out a survey some individuals may think, "What is the correct answer they want," while others may think, "How would I answer this question". The answers to some opinion questions may be different depending on which view an individual has when they answer the question leading to two underlying different patterns of response). At present, it is often not possible to tell the difference between a true skewed distribution and a mixture of two distributions which leads to a skewed joint distribution.
Due to the difficulty involved in computing combinations of multiple distributions, it wasn't until about the 1970s that these methods began to take off in a multitude of different fields and applications. The first real empirical example of this method was a study of the ratio of crab foreheads to their body length and the carapace of prawns (Pearson, 1894; Stigler, 1986, ch.10). The methodology and different approaches are multitudinous. It appears that mixtures of normals is most common, although examples are not hard to find of most of the other common statistical distributions. It appears that the maximum likelihood estimation method for determining the distributions is the most common now, although method of moment, kernel density, and a number of other methods are also fairly well covered in the literature.
To begin with, this bibliography will probably be like a meta-analysis where I introduce some of the general textbooks to the subject and the non-technical articles, encyclopedia entries, and review articles. Because the multitude of empirical articles may make it prohibitive to do a comprehensive, cross-disciplinary bibliography and review, I may not try to get copies and reference all of the articles that have been published on this topic, but will probably lean towards any social science applications first and biological or developmental ones next. I will also cover examples that are the most commonly discussed first. I am also more interested initially in continuous distributions than discrete ones. A review for discrete finite mixture modeling (Blischke, 1963) may provide a starting place for those interested in this.
Mixture modeling is extremely flexible and it optimally suited for understanding and modeling the diversity and complexity in data. It is my ultimate aim to gain a deep understanding of the finite mixture modeling approach so that I can show how it provides an exploratory (inductive) method, compared to Cobb's method based on catastrophe theory which provides a confirmatory (deductive) method to dealing with many of the same applications.
General overviews & applications (Böhning, 2000; Everitt & Hand, 1981; Frühwirth-Schnatter, 2006; Lindsay, 1995; McLachlan & Basford, 1988; McLachlan & Peel, 2000; Titterington, Smith, & Makov, 2000)
Human Genetics (Schork, Allison, & Thiel,
1996)
(Gupta & Huang, 1981; Holgersson & Jorner, 1978;
Redner & Walker, 1984)
Basic concepts of mixtures (Flury, Airoldi,
& Biber, 1992)
Introduction: schizophrenia, mortality by geography, smoking
self-report, and reoperation for heart valves (Everitt, 1996)
Encyclopedia summaries (Blischke, 1978; Everitt, 1985;
Titterington, 1996)
Maximum likelihood (McLahclan, 1982)
Survival & hazard modeling with mixture models (McLachlan
& McGiffin, 1994)
Botany
Marketing (Wedel & Kamakura, 1998, Ch.6-7)
Education
Crab & prawn sizes (Pearson, 1894)
Almost 100 examples of genetics modeling (Schork, Allison,
& Thiel, 1996)
Sex of Halibut fish (Hasmer, 1973), or Water Pipet birds (Flury,
Airoldi, & Biber, 1992)
Medicine
Re-operation rates for degenerate heart valves (Everitt, 1996; McLachlan & McGiffin, 1994)
Pattern Recognition
Zoology
Normal
Review (Holgersson & Jorner, 1978)
Two normals (John, 1970a)
Gamma Distribution
Dickinson, 1974; John, 1970b.
Beta
Poissan
Others
Univariate
Multivariate
Nonic equation-5 moments (Pearson, 1894)
Simplification: Charlier, 1906; Charlier & Wicksell,
1924; Doetsch, 1928; Strömgren, 1934; Rao, 1952 [sec. 8b.6];
Common variance (cubic equation-4 moments, conditional
minimum chi-square) method:
Compares the nonic with the common variance model (Cohen,
1967)
(Rao, 1948)
Performs less efficiently than MLE (Tan & Chang, 1972)
Lindsay's method:
Univariate normal mixture (Lindsay, 1989)
Bivariate normal mixture (Lindsay & Basak, 1991)
Multivariate normal mixtures - starting values for MLE (Lindsay & Basak, 1993)
Review (McLachlan, 1982; Redner & Walker,
1984)
Clustering applications (McLachlan, 1982; McLachlan &
Basford, 1988)
Comparison: Method of Moment vs. Maximum Likelihood Estimation
Tan & Chang, 1972
MoM Inferior to MLE (Tan & Chang, 1972; Fryer &
Robertson, 1972; etc.
Software Review
BINOMIX, C.A. MAN, MIX, BMDP, STATA
(Haughton, 1997)
EMMIX, Autoclass, binomix, C.A.MAN, MCLUST/EMCLUST, MGT, MIX,
Mixbin, Gompertz Mixtures, MPLUS, MULTIMIX, NORMIX, SNOB, Baysian & Markov (McLachlan
& Peel, 2000, appendix)
C.A.MAN (Böhning, 2000)
Hazard/Survival Analysis
Basic Introduction (McLachlan, 1994)
Fraction of patients cured of disease (Farewell, 1986)
Cancer survival (Phillips, Coldman, & McBride, 2002)
Risk of death from coronary heart disease (Wienke,
Christensen, Skytthe, & Yashin, 2002)
Insecticide used on flour beetles (Kuo & Peng, 2000)
Latent Class- Custering Models
Review (McLachlan & Basford, 1988)
Blitschke, W. R. 1963. Mixtures of discrete distributions. Proc. of the International Symposium on Classical and Contagious Discrete Distributions (pp.351-372). New York: Pergamon.
Blitschke, W. R. 1978. Mixtures of distributions. In International Encyclopedia of Statistics, Vol. 1., W.H. Kruskal & J. M. Tanur (Eds.). New York: The free Presss, pp.174-180.
Böhning, D. 2000. Computer-assisted analysis of mixtures and applications: Meta-analysis, disease mapping and others. New York: Chapman & Hall/CRC.
Dickinson, J. P. 1974. On the resolution of a mixture of observations from two gamma distributions by the method of maximum likelihood. Metrika, 21, 133-141.
Everitt, B. 1985. Mixture distributions. In Encyclopedia of Statistical Sciences, Vol. 5, S. Kotz and N. L. Johnson (Eds.). New York : Wiley, pp.559-569.
Everitt, B. S. 1996. An introduction to finite mixture distributions. Statistical Methods in Medical Research, 5, 107-127.
Everitt, B. S., & Hand, D. J. 1981. Finite Mixture Distributions. London: Chapman & Hall.
Farewell, V. T. 1986. Mixture models in survival analysis: Are they worth the risk? The Canadian Journal of Statistics, 14, 257-262.
Flury, B. D., Airoldi, Jean-Pierre, & Biber, J.-P. 1992. Gender identification of Water Pipits (Anthus spinoletta) using mixtures of distributions. Journal of Theoretical Biology, 158, 465-480.
Frühwirth-Schnatter, S. 2006. Finite mixture and Markov switching models. New York: Springer.
Gupta, A. K., & Huang, W. T. 1981. On mixtures of distributions: a survey and some new results on ranking and selection. Journal of the Indian Statistical Association B, 43, 245-290.
Hasmer, D. W. 1973. A comparison of iterative maximum-likelihood estimates of the parameters of a mixture of two normal distributions under three different types of sample. Biometrics, 29, 761-770.
Haughton, D. 1997. Packages for estimating finite mixtures: A review. The American Statistician, 51, 194-205.
Holgersson, M., & Jorner, U. 1978. Decomposition of a mixture into normal components: a review. International Journal of Biomedical Computing, 9, 367-392.
John, S. 1970a. On identifying the population of origin of each observation in a mixture of observations from two normal populations. Technometrics, 12, 553-563.
John, S. 1970b. On identifying the population of origin of each observation in a mixture of observations from two gamma populations. Technometrics, 12, 565-568.
Kuo, L., & Peng, F. 2000. A mixture-model approach to the analysis of survival data. In D. Dey, S. Ghosh, & B. Mallick (Eds.), Generalized Linear Models: A bayesian perspective (pp.255-270).
Lindsay, B. G. 1989. Moment matrices: Applications in mixtures. The Annals of Statistics, 17, 722-740.
Lindsay, B. G. 1995. Mixture models: Theory, geometry and applications, NSF-CBMS Regional Conference Series in Probability and Statistics, Vol. 5. Alexandria, Virginia: Institute of Mathematical Statistics and the American Statistical Association.
Lindsay, B. G., & Basak, P. 1991. On using bivariate moment equations in mixed normal problems. In V. P. Godambe (Ed.), Estimating Functions (pp.305-317). London: Oxford.
Lindsay, B. G., & Basak, P. 1993. Multivariate normal mixtures: A fast consistent method of moments. Journal of the American Statistical Association, 88, 468-476.
McLachlan, G. L. 1982. The classification and mixture maximum likelihood approaches to cluster analysis. In Handbook of Statistics, Vol. 2, P.R. Krishnaiah and L. Kanal (Eds.). Amsterdam: North-Holland, pp. 199-208.
McLachlan, G. L. 1994. On the role of finite mixture models in survival analysis. Statistical Methods in Medical Research, 3, 211-226.
McLachlan, G. L., & Basford, K. E. 1988. Mixture models: Inference and applications to clustering. New York: Marcel Dekker.
McLachlan, G. L., & McGiffin, D. C. 1994. On the role of finite mixture models in survival analysis. Statistical Methods in Medical Research, 3, 211-226.
McLachlan, G., & Peel, D. 2000. Finite mixture models. New York: John Wiley & Sons.
Pearson, K. 1894. Contributions to the mathematical theory of evolution. Philosophical Transactions of the Royal Society of Lonon, A, 185, 71-110.
Phillips, N., Coldman, A., & McBride, M. L. 2002. Estimating cancer prevalence using mixture models for cancer survival. Statistics in Medicine, 21, 1257-1270.
Redner, R. A., & Walker, H. F. 1984. Mixture densisities, maximum likelihood and the EM algorithm. SIAM Review, 26, 195-239.
Schork, N. J., Allison, D. B., & Thiel, B. 1996. Mixture distributions in human genetics research. Statistical Methods in Medical Research, 5, 155-178.
Stigler, S. M. 1986. The history of statistics: The measurement of Uncertainty before 1900. Cambridge, Massachusetts: Belknap. (Ch. 10: Pearson & Yule, pp.326-361.)
Tan, W. Y., & Chang, W. C. 1972. Some comparisons of the method of moments and the method of maximum likelihood in estimating parameters of a mixture of two normal densities. Journal of the American Statistical Association, 67, 702-708.
Titterington, D. M. 1996. Mixture distributions (update). In Encyclopedia of Statistical Sciences, Vol. 1, S. Kotz, N. L. Johnson, & D. Banks (Eds.). New York: Wiley, pp.399-407.
Titterington, D. M., Smith, A. F., & Makov, U. E. 1985. Statistical Analysis of Finite Mixture Distributions. New York: John Wiley & Sons.
Wienke, A., Christensen, K., Skytthe, A., & Yashin, A. I. 2002. Genetic analysis of cause of death in a mixture model of bivariate lifetime data. Statistical Modelling, 2, 89-102.
Last Modified: 08 Feb 2008
Copyright © Gary Horlacher