Course description

The second of a two-part sequence on: Probability, random variables, discrete and continuous distributions, order statistics, limit theorems, point and interval estimation, uniformly most powerful tests, likelihood ratio tests, chi-square and F tests, nonparametric tests. PREREQ: MATH 3326 and MATH 3352. Offered Even Years in the Spring Semester.

See Section on planned topics for more details

Class notes

  1. Introduction: Philosophy of Statistics
  2. Introduction to point estimation
  3. Maximum Likelihood Estimation
  4. Bayesian approach to parameter estimation
  5. Point Estimation: Principles and Theory

Midterm II Review

Planned topics (Spring 2026)

The course description and objectives provide very little direction for this course. A tentative list of course topics is provided below. In parenthesis is provided some supplementary reference chapters from relevant textbooks.

  • Parameter estimation
    • Method of moments estimators (Rice, 8.4)
    • Maximum likelihood estimation (Rice, 8.5)
    • Bayesian estimation (Rice, 8.6)
    • Sufficiency, minimal sufficiency, and the invariance principle (Rice 8.8, Pawitan 3.1–3.2, Pawitan 2.8–2.9)
  • Parameter uncertainty
    • Exact Intervals (e.g., Rice 8.5.3)
    • Revisiting Bayesian posteriors (Rice, 8.6)
    • Frequentist intervals
      • The Score function and Fisher’s Information / Observed Information (Pawitan 2.5)
      • Asymptotic properties of the MLE (Rice 8.5.2–8.5.3)
      • Wald intervals (Rice 8.5.2–8.5.3, Pawitan 3.3)
    • Likelihood based intervals (Pawitan 2.6)
      • Likelihood ratios
      • Invariance / minimal sufficiency of likelihood (Pawitan 2.9)
      • Profile likelihooods (Pawitan 3.4)
    • Cram'er-Rao Lower Bound (Rice 8.7)
    • Bias and variance of point estimates (Pawitan 5)
      • Reducing bias via Taylor series, jacknife, or bootstrap methods.
  • Hypothesis testing (Rice 9.1–9.5, Casella and Berger 8.3)
    • Neyman-Pearson vs Fisher
    • Uniformly most powerful tests
    • Duality of confidence intervals and hypothesis tests.
    • Non-parametric tests
    • Chi-square tests
    • Criticisms of confidence intervals (Pawitan 5.10)

Optional topics

  • Survey Sampling (Rice Chapter 7)
  • Empirical Bayes, Hierarchical Bayes
  • Computational Statistics
    • Visualization, advanced R topics, command line computing, etc.
    • Numeric optimization
    • Monte Carlo methods
    • Advanced Bayesian methods (Metropolis-Hastings, MCMC, Gibbs-sampling)
  • Introduction to time-series
  • Regression Theory + Nonlinear regression (Rice Chapter 14)
  • Gaussian Mixture Models + the EM algorithm

Additional Course Information:

Math 4450 Notes (Fall 2025)

  1. Chapter 1: Probability

  2. Chapter 2: Random Variables

  3. Chapter 3: Joint Distributions

  4. Chapter 4: Expected Values

  5. Chapter 5: Limit Theorems

  6. Chapter 6: Distributions Derived from the Normal Distribution

Homework and participation assignments

Please read the grading rubric before submitting homework.

Midterm

Final Exam

TODO: Update.

The final exam will be help in our regular classroom on Dec 18, 10:00 a.m. – 12:00 p.m. The final exam will be comprehensive, closed book.

Acknowledgements and License

This course and the code involved are made available with an MIT license. Some components follow a Creative Commons Attribution non-commercial license. A longer list of acknowledgments is available.