Add AR(p) and MA(q) models

[northern-64bit]

Here is the validation of the implementations working:

The AR model:

Implementation

    import numpy as np
    import pandas as pd
    import statsmodels.api as sm
    import time
    
    # Simulate an AR(1) process
    np.random.seed(0)  # For reproducibility
    n = 1000
    alpha = 0.5  # AR(1) coefficient
    sigma = 1  # Noise standard deviation
    ar_process = np.random.normal(0, sigma, n)
    for t in range(1, n):
        ar_process[t] += alpha * ar_process[t-1]
    
    # Fit models
    start_lsm = time.time()
    phi_lsm, c_lsm, sigma2_lsm = get_ar_weights_lsm(ar_process, 1)
    end_lsm = time.time()
    
    start_yw = time.time()
    phi_yw, c_yk, sigma2_yw = estimate_ar_weights_yule_walker(ar_process, 1)
    end_yw = time.time()
    
    start_sm_arima = time.time()
    model_sm_arima = sm.tsa.ARIMA(ar_process, order=(1, 0, 0)).fit()
    sigma2_sm_arima = model_sm_arima.resid.var()
    end_sm_arima = time.time()
    
    # Results comparison
    results_comparison = {
        "Custom LSM": {"Phi": phi_lsm, "Sigma2": sigma2_lsm, "Time": end_lsm - start_lsm},
        "Custom Yule-Walker": {"Phi": phi_yw, "Sigma2": sigma2_yw, "Time": end_yw - start_yw},
        "Statsmodels ARIMA": {"Phi": model_sm_arima.params[1], "Sigma2": sigma2_sm_arima, "Time": end_sm_arima - start_sm_arima}
    }
    
    results_comparison

Result

{'Custom LSM': {'Phi': 0.4907072990615945,
  'Sigma2': 0.9718186552765827,
  'Time': 0.0009710788726806641},
 'Custom Yule-Walker': {'Phi': 0.49369116106281674,
  'Sigma2': 0.9764331694286648,
  'Time': 0.002042531967163086},
 'Statsmodels ARIMA': {'Phi': 0.49153504493181266,
  'Sigma2': 0.9742868834709236,
  'Time': 0.07370138168334961}}

The MA model:

Implementation

np.random.seed(42)  # For reproducibility

# Parameters for the MA(1) process
theta_true = 0.6
sigma2_true = 1
n = 100

errors = np.random.normal(0, np.sqrt(sigma2_true), n)

# Generate the MA(1) process
data_ma = np.zeros(n)
for t in range(1, n):
    data_ma[t] = errors[t] + theta_true * errors[t-1]

theta_custom, sigma2_custom, errors_custom = fit_ma_model(data_ma, 1)

model_sm = sm.tsa.ARIMA(data_ma, order=(0,0,1))
results_sm = model_sm.fit()
theta_sm = results_sm.maparams
sigma2_sm = results_sm.params['sigma2']

theta_custom, sigma2_custom, theta_sm, sigma2_sm

Result

(array([0.63490588]),
 0.8143169190774796,
 array([0.62839426]),
 0.8133570727952485)

[JerBouma]

Thank you! Will review this around the 18th of March!

[JerBouma]

Thank you 🫡