STAT 639V: Topics in Statistics

General information

Text

Monahan (2001), Numerical Methods of Statistics, Cambridge University Press
Venables and Smith (2007), An Introduction to R (An HTML version is available here)

Homework Assignments and Solutions

Final Project

Computing

Useful R-related links:

• R homepage
• CRAN mirrors

• Timing different methods to solve linear systems (9/4/07)
• Fitting a logistic regression model using Newton-Raphson method (9/11/07)
• Using R to fit a mixture of normals (9/18/07)
• Rejection sampling (10/2/07)
• Variance reduction methods (Control and antithetic variates) (10/4/07)
• Importance sampling (10/9/07)
• Bootstrap (10/{18, 23}/ 07)
• Gibbs sampling (11/1/07)
• Metropolis-Hastings algorithm (11/8/07)
• Utility functions for MCMC output analysis
• Density estimation (11/13/07)
• Nonparametric smoothing (11/15/07)
• R graphics (11/20/07)

Important Dates

Outline

• Week 1 (8/21, 8/23)
  • Topics: Introduction; R; overview of computer arithmetic.
  • Reading and Computing: Read the first two chapters of Monahan. More details on computer arithmetic can be found in the article ``What Every Computer Scientist Should Know About Floating-Point Arithmetic''. (Originally published in the 1991 issue of ACM Computing Surveys, a PDF version can be downloaded via the library web page). If you are not already familar with R, start reading ``An Introduction to R''.
• Week 2 (8/28, 8/30)
  • Topics: Numerical linear algebra and Least Squares problems.
  • Reading and Computing: Read Chapter 3 and 5 of Monahan. The objective is not to understand every detail, but to get a general sense of the issues and the methods available. Explore and try to get familiar with the linear algebra tools provided in R.
• Week 3 (9/4, 9/6)
  • Topics: Monte Carlo experiments; Optimization.
  • Reading and Computing: More information on Monte Carlo experiments can be found in the book by Gentle (Appendix A). On optimization, read Chapter 8 of Monahan, or Chapter 2 of Givens.
• Week 4 (9/11, 9/13)
  • Topics: Optimization.
  • Reading and Computing: We will conclude our discussion of univariate optimization and start treating the multivariate case. Chapter 8 of Monahan or Chapter 2 of Givens cover all the material.
• Week 5 (9/18, 9/20)
  • Topics: Optimization.
  • Reading and Computing: Multivariate optimization, multiple local maxima. Stochastic optimization: simulated annealing. We will see in detail how to use "optim" in R to perform multivariate optimization. Simulated annealing is covered in Chapter 3 of Givens, in the context of discrete optimization, i.e. optimization of a function defined over a discrete set. Read Section 3.4.
• Week 6 (9/25, 9/27)
  • Topics: Simulation of Random Variables.
  • Reading and Computing: We will look at methods to simulate random variables from non-uniform distributions. In particular, we will consider the method of transformations and rejection sampling. Givens cover these topics in Section 6.2. We will also explore what is available in R to generate random variables from standard parametric families.
• Week 7 (10/2, 10/4)
  • Topics: Simulation of Random Variables and Monte Carlo Integration.
  • Reading and Computing: In our discussion of methods to simulate random variables, we will consider adaptive rejection sampling. This is covered in (sub-sub-) Section 6.2.3.2 of Givens. We will then talk about the Monte Carlo method to compute expected values, how to estimate Monte Carlo errors, and variance reduction techniques. This is covered in Givens, Section 6.3.
• Week 8 (10/9, 10/11)
  • Topics: Simulation of Random Variables and Monte Carlo Integration.
  • Reading and Computing: We will continue our discussion of Monte Carlo integration. This week we will cover importance sampling.
• Week 9 (10/16, 10/18)
  • Topics: The Bootstrap.
  • Reading and Computing: Bootstrap methods are covered in Chapter 9 in Givens and Hoeting. A good book-length tratment is contained in Davison and Hinkley, Bootstrap methods and their application.
• Week 10 (10/23, 10/25)
  • Topics: Bootstrap; Markov chain Monte Carlo.
  • Reading and Computing: We will conlude our treatment of the bootstrap, discussing bootstrap confidence intervals. Explore the recommended package "boot" and see how you can reproduce the examples we did in class using the functions of the package. We will also start discussing MCMC, which is covered in chapter 7 of Givens and Hoeting.
• Week 11 (10/30, 11/1)
  • Topics: MCMC, Gibbs sampling
  • Reading and Computing: Read section 7.2 of Givens and Hoetings.
• Week 12 (11/6, 11/8)
  • Topics: MCMC and metropolis-Hastings algorithm
  • Reading and Computing: Read section 7.1 and 7.3 of Givens and Hoeting.
• Week 13 (11/13, 11/15)
  • Topics: Density estimation and nonparametric smoothers
  • Reading and Computing: Read section 10.1 and 10.2 of Givens and Hoeting for density estimation; 11.2.3 and 11.2.5 for smoothing. Nonlinear smoothers like 'loess' and 'supersmoother', that we barely mentioned but are available in R, are treated in section 11.4
• Week 14 (11/20)
  • Topics: Graphics for data analysis
  • Reading and Computing: Chapter 4 of MASS has a lot of information on the graphic capabilities of R. A book-length treatment of the subject is R Graphics by Paul Murrell.

Lecture notes

• August 28