Course description

The second of a two-part sequence on: Probability, random variables, discrete and continuous distributions, order statistics, limit theorems, point and interval estimation, uniformly most powerful tests, likelihood ratio tests, chi-square and F tests, nonparametric tests. PREREQ: MATH 3326 and MATH 3352. Offered Even Years in the Spring Semester.

See Section on planned topics for more details

Class notes

  1. Introduction: Philosophy of Statistics
  2. Introduction to point estimation
  3. Maximum Likelihood Estimation
  4. Bayesian approach to parameter estimation

Planned topics (Spring 2026)

The course description and objectives provide very little direction for this course. A tentative list of course topics is provided below. In parenthesis is provided some supplementary reference chapters from relevant textbooks.

  • Parameter estimation
    • Method of moments estimators (Rice, 8.4)
    • Maximum likelihood estimation (Rice, 8.5)
    • Bayesian estimation (Rice, 8.6)
    • Sufficiency, minimal sufficiency, and the invariance principle (Rice 8.8, Pawitan 3.1–3.2, Pawitan 2.8–2.9)
  • Parameter uncertainty
    • Exact Intervals (e.g., Rice 8.5.3)
    • Revisiting Bayesian posteriors (Rice, 8.6)
    • Frequentist intervals
      • The Score function and Fisher’s Information / Observed Information (Pawitan 2.5)
      • Asymptotic properties of the MLE (Rice 8.5.2–8.5.3)
      • Wald intervals (Rice 8.5.2–8.5.3, Pawitan 3.3)
    • Likelihood based intervals (Pawitan 2.6)
      • Likelihood ratios
      • Invariance / minimal sufficiency of likelihood (Pawitan 2.9)
      • Profile likelihooods (Pawitan 3.4)
    • Cram'er-Rao Lower Bound (Rice 8.7)
    • Bias and variance of point estimates (Pawitan 5)
      • Reducing bias via Taylor series, jacknife, or bootstrap methods.
  • Hypothesis testing (Rice 9.1–9.5, Casella and Berger 8.3)
    • Neyman-Pearson vs Fisher
    • Uniformly most powerful tests
    • Duality of confidence intervals and hypothesis tests.
    • Non-parametric tests
    • Chi-square tests
    • Criticisms of confidence intervals (Pawitan 5.10)

Optional topics

  • Survey Sampling (Rice Chapter 7)
  • Empirical Bayes, Hierarchical Bayes
  • Computational Statistics
    • Visualization, advanced R topics, command line computing, etc.
    • Numeric optimization
    • Monte Carlo methods
    • Advanced Bayesian methods (Metropolis-Hastings, MCMC, Gibbs-sampling)
  • Introduction to time-series
  • Regression Theory + Nonlinear regression (Rice Chapter 14)
  • Gaussian Mixture Models + the EM algorithm

Additional Course Information:

Math 4450 Notes (Fall 2025)

  1. Chapter 1: Probability

  2. Chapter 2: Random Variables

  3. Chapter 3: Joint Distributions

  4. Chapter 4: Expected Values

  5. Chapter 5: Limit Theorems

  6. Chapter 6: Distributions Derived from the Normal Distribution

Homework and participation assignments

Please read the grading rubric before submitting homework.

Midterm

Final Exam

TODO: Update.

The final exam will be help in our regular classroom on Dec 18, 10:00 a.m. – 12:00 p.m. The final exam will be comprehensive, closed book.

Acknowledgements and License

This course and the code involved are made available with an MIT license. Some components follow a Creative Commons Attribution non-commercial license. A longer list of acknowledgments is available.