Citation & Reference

Sweep algorithm

  • Sweep based on SweepOperator.jl. Thanks to @joshday and @Hua-Zhou.



  • Section 7.4-7.6 of Numerical Analysis for Statisticians by Kenneth Lange (2010).

  • The paper A tutorial on the SWEEP operator by James H. Goodnight (1979).

REML & Parameter estimation

  • Henderson, C. R., et al. “The Estimation of Environmental and Genetic Trends from Records Subject to Culling.” Biometrics, vol. 15, no. 2, 1959, pp. 192–218. JSTOR,

  • Laird, Nan M., and James H. Ware. “Random-Effects Models for Longitudinal Data.” Biometrics, vol. 38, no. 4, 1982, pp. 963–974. JSTOR,

  • Lindstrom & J.; Bates, M. (1988). Newton—Raphson and EM Algorithms for Linear Mixed-Effects Models for Repeated-Measures Data. Journal of the American Statistical Association. 83. 1014. 10.1080/01621459.1988.10478693.

  • Gurka, Matthew. (2006). Selecting the Best Linear Mixed Model under REML. The American Statistician. 60. 19-26. 10.1198/000313006X90396.

  • Sang Hong Lee, Julius H.J. van der Werf. An efficient variance component approach implementing an average information REML suitable for combined LD and linkage mapping with a general complex pedigree. Genetics Selection Evolution, BioMed Central, 2006, 38 (1), pp.25-43. ⟨hal-00894558⟩

  • F.N. Gumedze, T.T. Dunne, Parameter estimation and inference in the linear mixed model, Linear Algebra and its Applications, Volume 435, Issue 8, 2011, Pages 1920-1944, ISSN 0024-3795, (

AI algorithm

  • D.L. Johnson, R. Thompson, Restricted Maximum Likelihood Estimation of Variance Components for Univariate Animal Models Using Sparse Matrix Techniques and Average Information, Journal of Dairy Science, Volume 78, Issue 2, 1995, Pages 449-456, ISSN 0022-0302, (

  • Mishchenko, Kateryna & Holmgren, Sverker & Rönnegård, Lars. (2007). Newton-type Methods for REML Estimation in Genetic Analysis of Quantitative Traits. Journal of Computational Methods in Science and Engineering. 8. 10.3233/JCM-2008-81-203.

  • Matilainen K, Mäntysaari EA, Lidauer MH, Strandén I, Thompson R. Employing a Monte Carlo algorithm in Newton-type methods for restricted maximum likelihood estimation of genetic parameters. PLoS One. 2013;8(12):e80821. Published 2013 Dec 10. doi:10.1371/journal.pone.0080821

Covariance structures

  • Wolfinger, Russ. (1993). Covariance structure selection in general mixed models. Communications in Statistics-simulation and Computation - COMMUN STATIST-SIMULAT COMPUT. 22. 1079-1106. 10.1080/03610919308813143.

  • Wolfinger, Russ. (1996). Heterogeneous Variance: Covariance Structures for Repeated Measures. Journal of Agricultural, Biological, and Environmental Statistics. 1. 205. 10.2307/1400366.

  • Littell, Ramon & Pendergast, Jane & Natarajan, Ranjini. (2000). Modelling covariance structure in the analysis of repeated measures data. Statistics in Medicine. 19. 1793-1819. 10.1002/1097-0258(20000715)19:13%3C1793::AID-SIM482%3E3.0.CO;2-Q.


  • McNeish, D., Harring, J. Covariance pattern mixture models: Eliminating random effects to improve convergence and performance. Behav Res 52, 947–979 (2020).

And more

  • Giesbrecht, F. G., and Burns, J. C. (1985), "Two-Stage Analysis Based on a Mixed Model: Large-sample Asymptotic Theory and Small-Sample Simulation Results," Biometrics, 41, 853-862.

  • Jennrich, R., & Schluchter, M. (1986). Unbalanced Repeated-Measures Models with Structured Covariance Matrices. Biometrics, 42(4), 805-820. doi:10.2307/2530695

  • Fletcher, Roger (1987), Practical methods of optimization (2nd ed.), New York: John Wiley & Sons, ISBN 978-0-471-91547-8

  • Wolfinger et al., (1994) Computing gaussian likelihoods and their derivatives for general linear mixed models doi: 10.1137/0915079

  • Hrong-Tai Fai & Cornelius (1996) Approximate F-tests of multiple degree of freedom hypotheses in generalized least squares analyses of unbalanced split-plot experiments, Journal of Statistical Computation and Simulation, 54:4, 363-378, DOI: 10.1080/00949659608811740

  • Schaalje GB, McBride JB, Fellingham GW. Adequacy of approximations to distributions of test statistics in complex mixed linear models. J Agric Biol Environ Stat. 2002;7:512–24.

  • Wright, Stephen, and Jorge Nocedal (2006) "Numerical optimization." Springer

  • Van Peer, A. (2010), Variability and Impact on Design of Bioequivalence Studies. Basic & Clinical Pharmacology & Toxicology, 106: 146-153. doi:10.1111/j.1742-7843.2009.00485.x

Julia packages

  • Revels, Jarrett & Lubin, Miles & Papamarkou, Theodore. (2016). Forward-Mode Automatic Differentiation in Julia.

  • Mogensen et al., (2018). Optim: A mathematical optimization package for Julia. Journal of Open Source Software, 3(24), 615,doi: 10.21105/joss.00615

CuSolver & CuBLAS



Reference dataset

  • Bioequivalence reference datasets: Schütz, H., Labes, D., Tomashevskiy, M. et al. Reference Datasets for Studies in a Replicate Design Intended for Average Bioequivalence with Expanding Limits. AAPS J 22, 44 (2020).

  • sleepstudy.csv: Gregory Belenky, Nancy J. Wesensten, David R. Thorne, Maria L. Thomas, Helen C. Sing, Daniel P. Redmond, Michael B. Russo and Thomas J. Balkin (2003) Patterns of performance degradation and restoration during sleep restriction and subsequent recovery: a sleep dose-response study. Journal of Sleep Research 12, 1–12.

  • Penicillin.csv: O.L. Davies and P.L. Goldsmith (eds), Statistical Methods in Research and Production, 4th ed., Oliver and Boyd, (1972), section 6.6

  • Pastes.csv: O.L. Davies and P.L. Goldsmith (eds), Statistical Methods in Research and Production, 4th ed., Oliver and Boyd, (1972), section 6.5

  • ChickWeight.csv:

    • Crowder, M. and Hand, D. (1990), Analysis of Repeated Measures, Chapman and Hall (example 5.3)
    • Hand, D. and Crowder, M. (1996), Practical Longitudinal Data Analysis, Chapman and Hall (table A.2)
    • Pinheiro, J. C. and Bates, D. M. (2000) Mixed-effects Models in S and S-PLUS, Springer.
  • RepeatedPulse.csv: Data supplied by a student at Oberlin College.

See also: