Regression and Time Series Model Selection

The following sections are included:

• Contents

• Preface

• List of Tables

The following sections are included:

• Background
  • Historical Review
  • Efficient Criteria
  • Consistent Criteria

• Overview
  • Distributions
  • Model Notation
  • Discrepancy and Distance Measures
  • Efficiency under Kullback–Leibler and L₂
  • Overfitting and Underfitting

• Layout

• Topics Not Covered

The following sections are included:

• Model Description
  • Model Structure and Notation
  • Distance Measures

• Derivations of the Foundation Model Selection Criteria

• Moments of Model Selection Criteria
  • AIC and AICc
  • FPE and Cp
  • SIC and HQ
  • Adjusted R²,

• Signal-to-noise Corrected Variants
  • AICu
  • FPEu
  • HQc

• Overfitting
  • Small-sample Probabilities of Overfitting
  • Asymptotic Probabilities of Overfitting
  • Small-sample Signal-to-noise Ratios
  • Asymptotic Signal-to-noise Ratios

• Small-sample Underfitting
  • Distributional Review
  • Expectations of L₂ and Kullback–Leibler Distance
  • Expected Values for Two Special Case Models
  • Signal-to-noise Ratios for Two Special Case Models
  • Small-sample Probabilities for Two Special Case Models

• Random X Regression and Monte Carlo Study

• Summary

• Appendix 2A. Distributional Results in the Central Case

• Appendix 2B. Proofs of Theorems 2.1 to 2.6

• Appendix 2C. Small-sample and Asymptotic Properties

• Appendix 2D. Moments of the Noncentral χ²

The following sections are included:

• Model Description
  • Autoregressive Models
  • Distance Measures

• Selected Derivations of Model Selection Criteria
  • AIC
  • AICc
  • AICu
  • FPE
  • FPEu
  • Cp
  • SIC
  • HQ
  • HQc

• Small-sample Signal-to-noise Ratios

• Overfitting
  • Small-sample Probabilities of Overfitting
  • Asymptotic Probabilities of Overfitting
  • Small-sample Signal-to-noise Ratios
  • Asymptotic Signal-to-noise Ratios

• Underfitting for Two Special Case Models
  • Expected Values for Two Special Case Models
  • Signal-to-noise Ratios for Two Special Case Models
  • Probabilities for Two Special Case Models

• Autoregressive Monte Carlo Study

• Moving Average MA(1) Misspecified as Autoregressive Models
  • Two Special Case MA(1) Models
  • Model and Distance Measure Definitions
  • Expected Values for Two Special Case Models
  • Misspecified MA(1) Monte Carlo study

• Multistep Forecasting Models
  • Kullback–Leibler Discrepancy for Multistep
  • AICcm, AICm, and FPEm
  • Multistep Monte Carlo Study

• Summary

• Appendix 3A. Distributional Results in the Central Case

• Appendix 3B. Small-sample Probabilities of Overfitting

• Appendix 3C. Asymptotic Results

The following sections are included:

• Model Description
  • Model Structure and Notation
  • Distance Measures

• Selected Derivations of Model Selection Criteria
  • L₂-based Criteria FPE and Cp
  • Kullback–Leibler-based Criteria AIC and AICc
  • Consistent Criteria SIC and HQ

• Moments of Model Selection Criteria
  • AIC and AICc
  • SIC and HQ

• Signal-to-noise Corrected Variants
  • AICu
  • HQc

• Overfitting Properties
  • Small-sample Probabilities of Overfitting
  • Asymptotic Probabilities of Overfitting
  • Asymptotic Signal-to-noise Ratio

• Underfitting
  • Distributions for Underfitted Models
  • Expected Values for Two Special Case Models
  • Signal-to-noise Ratios for Two Special Case Models
  • Probabilities for Two Special Case Models

• Monte Carlo Study

• Summary

• Appendix 4A. Distributional Results in the Central Case

• Appendix 4B. Proofs of Theorems 4.1 to 4.5

• Appendix 4C. Small-sample Probabilities of Overfitting

• Appendix 4D. Asymptotic Probabilities of Overfitting

• Appendix 4E. Asymptotic Signal-to-noise Ratios

The following sections are included:

• Model Description
  • Vector Autoregressive Models
  • Distance Measures

• Selected Derivations of Model Selection Criteria
  • FPE
  • AIC
  • AICc
  • AICu
  • SIC
  • HQ
  • HQc

• Small-sample Signal-to-noise Ratios
  • AIC
  • AICc
  • AICu
  • SIC
  • HQ
  • HQc

• Overfitting
  • Small-sample Probabilities of Overfitting
  • Asymptotic Probabilities of Overfitting
  • Asymptotic Signal-to-noise Ratios

• Underfitting in Two Special Case Models
  • Expected Values for Two Special Case Models
  • Signal-to-noise Ratios for Two Special Case Models
  • Probabilities for Two Special Case Models

• Vector Autoregressive Monte Carlo Study

• Summary

• Appendix 5A. Distributional Results in the Central Case

• Appendix 5B. Small-sample Probabilities of Overfitting

• Appendix 5C. Asymptotic Probabilities of Overfitting

• Appendix 5D. Asymptotic Signal-to-noise Ratios

The following sections are included:

• Univariate Regression Cross-validation
  • Withhold-1 Cross-validation
  • Delete-d Cross-validation

• Univariate Autoregressive Cross-validation
  • Withhold-1 Cross-validation
  • Delete-d Cross-validation

• Multivariate Regression Cross-validation
  • Withhold-1 Cross-validation
  • Delete-d Cross-validation

• Vector Autoregressive Cross-validation
  • Withhold-1 Cross-validation
  • Delete-d Cross-validation

• Univariate Regression Bootstrap
  • Overview of the Bootstrap
  • Doubly Cross-validated Bootstrap Selection Criterion

• Univariate Autoregressive Bootstrap

• Multivariate Regression Bootstrap

• Vector Autoregressive Bootstrap

• Monte Carlo Study
  • Univariate Regression
  • Univariate Autoregressive Models

• Summary

The following sections are included:

• Nonnormal Error Regression Models
  • L₁ Distance and Efficiency

• Least Absolute Deviations Regression
  • L1AICc
  • Special Case Models

• Robust Version of Cp
  • Derivation of RCp

• Wald Test Version of Cp

• FPE for Robust Regression

• Unification of AIC Criteria
  • The Unification of the AIC Family
  • Location–Scale Regression Models
  • Monte Carlo Study

• Quasi-likelihood
  • Selection Criteria for Extended Quasi-likelihood Models
  • Quasi-likelihood Monte Carlo Study

• Summary

• Appendix 7A. Derivation of AICc under Quasi-likelihood

The following sections are included:

• Model Selection in Nonparametric Regression
  • AIC for Smoothing Parameter Selection
  • Nonparametric Monte Carlo Study

• Semiparametric Regression Model Selection
  • The Family of Candidate Models
  • AICc
  • Semiparametric Monte Carlo Study

• A Cross-validatory AIC for Hard Wavelet Thresholding
  • Wavelet Reconstruction and Thresholding
  • Nason's Cross-validation Method
  • Cross-validatory AICc
  • Properties of the AICc Selected Estimator
  • Wavelet Monte Carlo Study

• Summary

The following sections are included:

• Introduction
  • Univariate Criteria List
  • Multivariate Criteria List
  • Nonparametric Rank Test for Criteria Comparison

• Univariate Regression Models
  • Model Structure
  • Special Case Models
  • Large-scale Small-sample Simulations
  • Large-sample Simulations
  • Real Data Example

• Autoregressive Models
  • Model Structure
  • Two Special Case Models
  • Large-scale Small-sample Simulations
  • Large-sample Simulations
  • Real Data Example

• Moving Average MA(1) Misspecified as Autoregressive Models
  • Model Structure
  • Two Special Case Models
  • Large-scale Small-sample Simulations
  • Large-sample Simulations

• Multivariate Regression Models
  • Model Structure
  • Two Special Case Models
  • Large-scale Small-sample Simulations
  • Large-sample Simulations
  • Real Data Example

• Vector Autoregressive Models
  • Model Structure
  • Two Special Case Models
  • Large-scale Small-sample Simulations
  • Large-sample Simulations
  • Real Data Example

• Summary

• Appendix 9A. Details of Simulation Results

• Appendix 9B. Stepwise Regression

The following sections are included:

• References

• Author Index

• Index