Buy Bayesian Ideas and Data Analysis
Buy Bayesian Ideas and Data Analysis
Chapter 13 of this contains newer computing discussion.
Bayesian statistics is about probability. What is the probability that a new chemotherapy treatment will be effective? What is the probability that a particular type of bank will go bankrupt? What is the probability that a positive mammogram is truly indicative of breast cancer? The Bayesian method combines expert scientific information with data using Bayes Theorem to obtain such probabilities. Dennis Lindley, a famous Bayesian, has asserted that there are two rules in Bayesian inference: (i) all uncertainty is modeled using probability and (ii) always obey the laws of probability. Consequently, a primary prerequisite for this course in a course in probability, preferably a calculus based course.
For most of what we do, probability is simply the area (or volume) under a curve (or surface) over a set. Such curves (surfaces) are defined by probability density functions. Bayesian inference requires integrating particular functions against density functions, for example, to obtain a prediction or the mean or variance of a quantity of interest. Sometimes, it is possible to evaluate integrals analytically, and we start the book with problems where the calculus is tractable. However, since most integrals that are necessary for Bayesian computation are not tractable, we turn to numerical approximation by simulation, in particular Markov chain Monte Carlo (MCMC) simulation. Appendices A and B present the basics of matrix algebra and probability for those whose skills in these areas are rusty.
Of course the more statistics courses you have had the better. We introduce a very large number of statistical models in the book, and the more of them that you have seen in other courses, the better. However, some of us have taught a version of this course to individuals who have only had a single probability course, and another version to scientifically sophisticated students who have not had a calculus based probability course, but who did have a background involving one or more applied statistics courses.
The first five chapters of the book constitute our version of the core of a traditional Bayesian Statistics course, covering the basics of Bayesian ideas, calculations and inference, including modeling one and two sample data from traditional normal, binomial, Poisson, exponential and other sampling models. Chapter 1 is motivational, presenting a number of portraits of the power of the Bayesian approach based on scientific scenarios that we have previously encountered and subsequently address in the book. Chapter 2 presents the fundamentals of Bayesian philosophy, and its' implementation including {\it real} prior specification, data modeling, and posterior inferences via Bayes Theorem. Chapter 3 presents the interplay between probability calculus and its approximation by simulation, and the implementation of simulation via WinBUGS. Chapter 4 develops deeper foundational issues, including aspects of hypothesis testing, exchangeability, prediction, model checking and selection, ``diffuse" prior specification, large sample approximation to posteriors, consistency, identifiability, and hierarchical modeling. Chapter 5 handles one and two sample analysis of binomial data, including relative risk estimation, as well as inferences based on normal and Poisson sampling. Also included in Chapter 5 is a treatment of case-control sampling and traditional odds ratio estimation, and methods for sample size determination.
Chapter 6 discusses the theoretical basis for Markov chain Monte Carlo simulation, and issues related to its practical application.
Chapter 7 introduces the concept of regression modeling at an elementary level, and Chapter 8 covers binomial regression including logistic regression for correlated data using generalized linear mixed models. Chapter 9 presents methods for the general linear model (analysis of variance and regression) and Chapter 10 extends those methods to handle correlated clustered and longitudinal measurement data using linear mixed models and multivariate normal analysis. Chapter 11 carries on the discussion of generalized linear models with coverage of Poisson regression including mixed models.
Chapters 12 covers the topic of time to event data (survival analysis) based on one and two sample data that are subject to censoring. Chapter 13 continues by developing regression models for survival data, including the accelerated failure time model and the Cox proportional hazards model. The chapter concludes with a discussion of frailty models for correlated survival data.
Chapter 14 discusses the topic of binary diagnostic testing. Material on continuous-response diagnostic test data appears on the book website.
Finally, Chapter 15 covers semiparametric and nonparametric inference. This chapter covers density estimation and flexible regression modeling of mean functions. %, thus allowing for curves as a function of time in longitudinal modeling for example, and taking standard parametric regression models and %allowing for great flexibility in modeling.
We started this project with the intention of writing a compact book. We ended it with enough material for two books (Chapters 1-9 and 11 would make up volume 1 with the remaining chapters being a second volume). There are a number of different versions of courses that we envision from our book. The most elementary chapters are 1, 2, 3, 5, 7, 11 and 14. If students have already had a course in regression modeling, Chapter 7 would not be necessary, but could be treated as required reading nonetheless. The most sophisticated chapters are 4, 6, 13 and 15. For more elementary courses we envision instructor discussion of parts of Chapter 4 on an {\it as needed} basis when covering subsequent chapters. Chapters 4 and 6 would be a must for more advanced courses. Different versions of this course that we envision are:
Exercises are an integral part of the book. Even if students choose not to do them, they should read them. They are often strategically placed in order to reinforce the surrounding discussion.
While our book is written with an emphasis on the use of WinBUGS and R, it is certainly feasible to teach a Bayesian course without requiring either of these software packages. For instance, we illustrate the use of SAS in some examples. Historically, books have been written either without thought of analyzing real data, which was one of our motivations for writing a different kind of book, or authors envisioned data analysis but left open the question of what software might be used. %In fact, we have previously used Gauss and R software in previous versions of this course. Instructors should have no difficulty using our book if that is the case. It is of course important for statistics Ph.D. students to learn to write their own programs for analyzing data. Some universities teach Bayesian computing in a separate course. When this isn't the case, instructors can easily augment the course with assignments to write code in the language of their choice. Much of our WinBUGS code serves as a kind of precise description of modeling efforts and the types of inferences we intend to make, which can be translated into other environments like R or Matlab.
The seeds for this book were planted when the first two authors were graduate students at the University of Minnesota starting in 1974. The environment there, which was created in large part by Seymour Geisser, fostered and promoted the foundational aspects of Statistics in virtually all aspects of the program. Surprisingly, there were no Bayesian courses in their curriculum at that time, but Bayesian ideas and methods were an integral part of many key courses. So we ``grew up" thinking that Bayesian methods were just another way to approach the whole of inferential statistics.
Another phase of development devolved from efforts with Ed Bedrick on how to specify what we call ``real" priors, that allow for incorporation of input from subject matter experts in generalized linear models. Indeed, a unique aspect of this book is its' emphasis on incorporating ``real" prior information.
From the early 1980s, Statistics 145 at UC Davis was taught initially by the second author as a successful and purely theoretical course using the seminal book by Jim Berger (***). It evolved through the early 1990s, during a failed attempt to make the course accessible to non-statistics graduate students, and then finally succeeded with a broad audience, including Statistics students, in our opinion due to the advent of the WinBUGS software (Spiegelhalter et al., ****). A missing component in previous versions was the capability of analyzing a wide variety of simple to complex data sets on top of emphasizing the foundations of the subject.
The final version of the book was transformed by joint efforts that were lead by the first author, and which were ultimately guided by the influence of Seymour Geisser and Spiegelhalter et al. Indeed, we have emphasized the importance of prediction and ease of computation throughout.