Model selection including sparsity methods and their oracle properties, information methods, cross-validation and stochastic search. Basic theory of kernel methods for regression. Classification: linear and quadratic discriminants, Bayes classifier, nearest neighbor methods, kernel methods for classification. Introduction to neural networks and recursive partitioning. Model averaging methods and measures of complexity. Cluster analysis.