Time Series for Data Science: Analysis and Forecasting

Cover
Half Title
Series Information
Title Page
Copyright Page
Dedication
Table of Contents
Preface
Acknowledgments
Authors
1 Working With Data Collected Over Time
1.1 Introduction
1.2 Time Series Datasets
1.2.1 Cyclic Data
1.2.1.1 Sunspot Data
1.2.1.2 DFW Temperature Data
1.2.1.3 Air Passengers Data
1.2.2 Trends
1.2.2.1 Real Datasets That Have Trending Behavior
1.2.2.2 The Problem With Trends
1.3 The Programming Language R
1.3.1 The Tswge Time Series Package
1.3.2 Base R
1.3.3 Plotting Time Series Data in R
1.3.4 The Ts Object
1.3.4.1 Creating a Ts Object
1.3.4.2 More About Ts Objects
1.3.5 The Plotts.wge Function in Tswge
1.3.5.1 Modifying the Appearance of Plots Using the Tswge Plotts.wge Function
1.3.6 Loading Time Series Data Into R
1.3.6.1 The .csv File
1.3.6.2 The .txt File
1.3.6.3 Other File Formats
1.3.7 Accessing Time Series Data
1.3.7.1 Accessing Data From the Internet
1.3.7.2 Business / Proprietary Data: Ozona Bar and Grill
1.4 Dealing With Messy Data
1.4.1 Preparing Time Series Data for Analysis: Cleaning, Wrangling, and Imputation
1.4.1.1 Missing Data
1.4.1.2 Downloading When No .csv Download Option Is Available
1.4.1.3 Data That Require Cleaning and Wrangling
1.4.1.4 Programatic Method of Ingestion and Wrangling Data From Tables On Web Pages
1.5 Concluding Remarks
Appendix 1A
tswge Datasets INTRODUCED IN this Chapter
Problems
2 Exploring Time Series Data
2.1 Understanding and Visualizing Data
2.1.1 Smoothing Time Series Data
2.1.1.1 Smoothing Data Using a Centered Moving Average Smoother
2.1.1.2 Other Methods Available for Smoothing Data
2.1.1.3 Moving Average Smoothing Versus Aggregating
2.1.1.4 Using Moving Average Smoothing for Estimating Trend in Data With Fixed Cycle Lengths
2.1.2 Decomposing Seasonal Data
2.1.2.1 Additive Decompositions
2.1.2.2 Multiplicative Decompositions
2.1.3 Seasonal Adjustment
2.1.3.1 Additive Seasonal Adjustment
2.1.3.2 Multiplicative Seasonal Adjustment
2.2 Forecasting
2.2.1 Predictive Moving Average Smoother
2.2.2 Exponential Smoothing
2.2.2.1 Forecasting With Exponential Smoothing Beyond the Observed Dataset
2.2.3 Holt-Winters Forecasting
2.2.3.1 Additive Holt-Winters Equations
2.2.3.2 Multiplicative Holt-Winters Equations
2.2.4 Assessing the Accuracy of Forecasts
2.3 Concluding Remarks
Appendix 2A
tswge Functions
Tswge Datasets Introduced In Chapter 2
3 Statistical Basics for Time Series Analysis
3.1 Statistics Basics
3.1.1 Univariate Data
3.1.2 Multivariate Data
3.1.2.1 Measuring Relationships Between Two Random Variables in a Bivariate Random Sample
3.1.2.2 Assessing Association From a Bivariate Random Sample
3.1.3 Independent Vs Dependent Data
3.2 Time Series and Realizations
3.2.1 Multiple Realizations
3.2.1.1 Time Series 1: Xt
3.2.1.2 Time Series 2: Yt (Example 3.3 Continued)
3.2.2 The Effect of Realization Length
3.3 Stationary Time Series
3.3.1 Plotting the Autocorrelations of a Stationary Process
3.3.2 Estimating the Parameters of a Stationary Process
3.3.2.1 Estimating
3.3.2.2 Estimating the Variance
3.3.2.3 Estimating the Autocovariance and Autocorrelation
3.3.2.4 Plotting Sample Autocorrelations
3.4 Concluding Remarks
Appendix 3A
Appendix 3B
TSWGE FUNCTION
BASE R COMMANDS
TSWGE DATASETS RELATED TO THIS CHAPTER
Problems
4 The Frequency Domain
4.1 Trigonometric Review and Terminology
4.2 The Spectral Density
4.2.1 Euler’s Formula
4.2.2 Definition and Properties of the Spectrum and Spectral Density
4.2.2.1 The Nyquist Frequency
4.2.2.2 Frequency F.=.0
4.2.2.3 The Spectral Density and the Autocorrelation Function
4.2.3 Estimating the Spectral Density
4.2.3.1 The Sample Spectral Density
4.2.3.2 Smoothing the Sample Spectral Density
4.2.3.3 Parzen Spectral Density Estimate Vs Sample Autocorrelations
4.2.3.4 Why We Plot Spectral Densities in Log Scale
4.3 Smoothing and Filtering
4.3.1 Types of Filters
4.3.2 The Butterworth Filter
4.4 Concluding Remarks
Appendix 4A
tswge Functions
Problems
5 ARMA Models
5.1 The Autoregressive Model
5.1.1 The AR(1) Model
5.1.1.1 The AR(1) in Backshift Operator Notation
5.1.1.2 The AR(1) Characteristic Polynomial and Characteristic Equation
5.1.1.3 Properties of a Stationary AR(1) Model
5.1.1.4 Spectral Density of an AR(1)
5.1.1.5 AR(1) Models With Positive Roots of the Characteristic Equation
5.1.1.6 AR(1) Models With Roots Close to +1
5.1.1.7 AR(1) Models With Negative Roots of the Characteristic Equation
5.1.1.8 Nonstationary 1st-Order Models
5.1.1.9 Final Comments Regarding AR(1) Models
5.1.2 The AR(2) Model
5.1.2.1 Facts About the AR(2) Model
5.1.2.2 Operator Notation and Characteristic Equation for an AR(2)
5.1.2.3 Stationary AR(2) With Two Real Roots
5.1.2.4 Stationary AR(2) With Complex Conjugate Roots
5.1.2.5 Summary of AR(1) and AR(2) Behavior
5.1.3 The AR(p) Models
5.1.3.1 Facts About the AR(p) Model
5.1.3.2 Operator Notation and Characteristic Equation for an AR(p)
5.1.3.3 Factoring the AR(p) Characteristic Polynomial
5.1.3.4 Factor Tables for AR(p) Models
5.1.3.5 Dominance of Roots Close to the Unit Circle
5.1.4 Linear Filters, the General Linear Process, and AR(p) Models
5.1.4.1 AR(1) in GLP Form
5.1.4.2 AR(p) in GLP Form
5.2 Autoregressive-Moving Average (ARMA) Models
5.2.1 Moving Average Models
5.2.1.1 The MA(1) Model
5.2.1.2 The MA(2) Model
5.2.1.3 The General MA(q) Model
5.2.1.4 Invertibility
5.2.2 ARMA(p,q) Models
5.2.2.1 Stationarity and Invertibility of an ARMA(p,q) Process
5.2.2.2 AR Versus ARMA Models
5.3 Concluding Remarks
Appendix 5A
tswge Functions
Appendix 5B
Stationarity Conditions of an AR(1)
Problem Set
6 ARMA Fitting and Forecasting
6.1 Fitting ARMA Models to Data
6.1.1 Estimating the Parameters of an ARMA(p,q) Model
6.1.1.1 Maximum Likelihood Estimation of the and Coefficients of an ARMA Model
6.1.1.2 Estimating .
6.1.1.3 Estimating
6.1.1.4 Alternative Estimates for AR(p) Models
6.1.2 ARMA Model Identification
6.1.2.1 Plotting the Data and Checking for White Noise
6.1.2.2 Model Identification Types
6.1.2.3 AIC-Type Measures for ARMA Model Fitting
6.1.2.4 The Special Case of AR Model Identification
6.2 Forecasting Using an ARMA(p,q) Model
6.2.1 ARMA Forecasting Setting, Notation, and Strategy
6.2.1.1 Strategy and Notation
6.2.1.2 Forecasting for
6.2.1.3 Forecasting At0+l for l > 0
6.2.2 Forecasting Using an AR Model
6.2.2.1 Forecasting Using an AR(1) Model
6.2.3 Basic Formula for Forecasting Using an ARMA() Model
6.2.4 Eventual Forecast Functions
6.2.5 Probability Limits for ARMA Forecasts
6.2.5.1 Facts About Forecast Errors
6.2.5.2 Lack of Symmetry
6.2.6 Assessing Forecast Performance
6.2.6.1 How “Good” Are the Forecasts?
6.2.6.2 Some Strategies for Using RMSE to Measure Forecast Performance
6.3 Concluding Remarks
Appendix 6A
tswge Functions
Problems
7 ARIMA and Seasonal Models
7.1 ARIMA(p, D, Q) Models
7.1.1 Properties of the ARIMA(p,d,q) Model
7.1.1.1 Some ARIMA() Models
7.1.1.2 Characteristic Equations for Models (A) . (C)
7.1.1.3 Limiting Autocorrelations
7.1.1.4 Lack of Attraction to a Mean
7.1.1.5 Random Trends
7.1.1.6 Differencing an ARIMA(0,1,0) Model
7.1.1.7 ARIMA Models With Stationary and Nonstationary Components
7.1.1.8 The Stationary AR(2) Model: (1-1.4B+.65B2)Xt = at
7.1.2 Model Identification and Parameter Estimation of ARIMA(p,d,q) Models
7.1.2.1 Deciding Whether to Include One Or More 1–.B Factors (That Is, Unit Roots) in the Model
7.1.2.2 General Procedure for Fitting an ARIMA(p,d,q) Model to a Set of Time Series Data
7.1.3 Forecasting With ARIMA Models
7.1.3.1 ARMA Forecast Formula
7.2 Seasonal Models
7.2.1 Properties of Seasonal Models
7.2.1.1 Some Seasonal Models
7.2.2 Fitting Seasonal Models to Data
7.2.2.1 Overfitting
7.2.3 Forecasting Using Seasonal Models
7.3 ARCH and GARCH Models
7.3.1 ARCH(1) Model
7.3.2 The ARCH(p) and GARCH(p,q) Processes
7.3.3 Assessing the Appropriateness of an ARCH/GARCH Fit to a Set of Data
7.3.4 Fitting ARCH/GARCH Models to Simulated Data
7.3.5 Modeling Daily Rates of Return Data
7.4 Concluding Remarks
Appendix 7A
tswge Functions
Appendix 7B
Problems
8 Time Series Regression
8.1 Line+Noise Models
8.1.1 Testing for Linear Trend
8.1.1.1 Testing for Trend Using Simple Linear Regression
8.1.1.2 A T-Test Simulation
8.1.1.3 Cochrane-Orcutt Test for Trend
8.1.1.4 Bootstrap-Based Test for Trend
8.1.1.5 Other Methods for Testing for Trend in Time Series Data
8.1.2 Fitting Line+Noise Models to Data
8.1.3 Forecasting Using Line+Noise Models
8.2 Cosine Signal+Noise Models
8.2.1 Fitting a Cosine Signal+Noise Model to Data
8.2.2 Forecasting Using Cosine Signal+Noise Models
8.2.2.1 Using Fore.sigplusnoise.wge
8.2.3 Deciding Whether to Fit a Cosine Signal+Noise Model to a Set of Data
8.2.3.1 A Closer Look at the Cyclic Behavior
8.3 Concluding Remarks
Appendix 8A
tswge Functions
Exercises
9 Model Assessment
9.1 Residual Analysis
9.1.1 Checking Residuals for White Noise
9.1.1.1 Check Residual Sample Autocorrelations Against 95% Limit Lines
9.1.1.2 Ljung-Box Test
9.1.2 Checking the Residuals for Normality
9.2 CASE STUDY 1: Modeling the Global Temperature Data
9.2.1 A Stationary Model
9.2.1.1 Checking the Residuals
9.2.1.2 Realizations and Their Characteristics
9.2.1.3 Forecasting Based On the ARMA(4,1) Model
9.2.2 A Correlation-Based Model With a Unit Root
9.2.2.1 Checking the Residuals
9.2.2.2 Realizations and Their Characteristics
9.2.2.3 Forecasting Based On ARIMA(0,1,1) Model
9.2.3 Line+Noise Models for the Global Temperature Data
9.2.3.1 Checking the Residuals, for White Noise
9.2.3.2 Realizations and Their Characteristics
9.2.3.3 Forecasting Based On the Signal-Plus-Noise Model
9.2.3.4 Other Forecasts
9.3 CASE STUDY 2: Comparing Models for the Sunspot Data
9.3.1 Selecting the Models for Comparison
9.3.2 Do the Models Whiten the Residuals?
9.3.3 Do Realizations and Their Characteristics Behave Like the Data?
9.3.4 Do Forecasts Reflect What Is Known About the Physical Setting?
9.3.4.1 Final Comments About the Models Fit to the Sunspot Data
9.4 Comprehensive Analysis of Time Series Data: A Summary
9.5 Concluding Remarks
Appendix 9A
tswge Function
Base R Function
Problems
10 Multivariate Time Series
10.1 Introduction
10.2 Multiple Regression With Correlated Errors
10.2.1 Notation for Multiple Regression With Correlated Errors
10.2.2 Fitting Multiple Regression Models to Time Series Data
10.2.2.1 Including a Trend Term in the Multiple Regression Model
10.2.2.2 Adding Lagged Variables
10.2.2.3 Using Lagged Variables and a Trend Variable
10.2.3 Cross Correlation
10.3 Vector Autoregressive (VAR) Models
10.3.1 Forecasting With VAR(p) Models
10.3.1.1 Univariate Forecasts
10.3.1.2 VAR Analysis
10.3.1.3 Comparing RMSEs
10.3.1.4 Final Comments
10.4 Relationship Between MLR and VAR Models
10.5 A Comprehensive and Final Example: Los Angeles Cardiac Mortality
10.5.1 Applying the VAR(p) to the Cardiac Mortality Data
(1) Use VARselect to Identify Candidate Model Orders
(2) Use VAR to Fit the VAR Models to the Training Set
(3) Use Predict to Forecast Data Values in the Test Set
10.5.2 The Seasonal VAR(p) Model
(1) Use VARselect to Identify Candidate Seasonal Models
(2) Use VAR to Fit the VAR Models to the Training Set
(3) Use Predict to Forecast Data Values in the Test Set
10.5.3 Forecasting the Future
10.5.3.1 Short Vs. Long Term Forecasts
10.6 Conclusion
Appendix 10A
BASE R FUNCTIONS
CRAN Package Dpylr
CRAN Package Vars
tswge Datasets Introduced in this Chapter
Appendix 10B
Relationship Between MLR With Correlated Errors and VAR
Three Important Points Should Be Considered
Problems
11 Deep Neural Network-Based Time Series Models
11.1 Introduction
11.2 The Perceptron
11.3 The Extended Perceptron for Univariate Time Series Data
11.3.1 A Neural Network Similar to the AR(1)
11.3.1.1 The Architecture
11.3.1.2 Fitting the MLP
11.3.1.3 Forecasting
11.3.1.4 Cross Validation Using the Rolling Window RMSE
11.3.2 A Neural Network Similar to AR(p): Adding More Lags
11.3.3 A Deeper Neural Network: Adding a Hidden Layer
11.3.3.1 Differences and Seasonal “Dummies”
11.4 The Extended Perceptron for Multivariate Time Series Data
11.4.1 Forecasting Melanoma Using Sunspots
11.4.1.1 Architecture
11.4.1.2 Fitting the Baseline Model
11.4.1.3 Forecasting Future Sunspot Data for Predicting Future Melanoma
11.4.1.4 Forecasting the Last Eight Years of Melanoma
11.4.1.5 Fitting a Competing Model
11.4.1.6 Assessing the Competing Model On the Last Eight Years of Melanoma Data
11.4.1.7 Forecasting the Next Eight Years of Melanoma
11.4.2 Forecasting Cardiac Mortality Using Temperature and Particulates
11.4.2.1 General Architecture
11.4.2.2 Train / Test Split
11.4.2.3 Forecasting Covariates: Temperature and Particulates
11.4.2.4 Model Without Seasonal Indicator Variables
11.4.2.5 Model With Seasonal Indicator Variables
11.5 An “Ensemble” Model
11.5.1 Final Forecasts for the Next Fifty-Two Weeks
11.5.2 Final Forecasts for the Next Three Years (Longer Term Forecasts)
11.6 Concluding Remarks
Appendix 11A
tswge Function
Appendix 11B
Chapter 11 Problems
Mini Research Project
References
Index