| Signature | Description |
|---|---|
enum class dw_autocorr_t : unsigned char { positive_autocorr = 1, // d < dL: reject H0 in favour of +ve AR(1) negative_autocorr = 2, // d > 4−dL: reject H0 in favour of −ve AR(1) no_autocorr = 3, // dU < d < 4−dU: fail to reject H0 inconclusive = 4, // dL <= d <= dU or 4−dU <= d <= 4−dL }; |
Interpretation of the Durbin-Watson d statistic. The Durbin-Watson bounds test compares d against critical values dL and dU that depend on n (sample size), k (number of regressors excluding the intercept), and the chosen significance level. Rather than shipping the full table of critical values inside the visitor, DurbinWatsonVisitor returns one of these four classifications based on the commonly-used rule-of-thumb thresholds (see get_result_category() documentation). |
| Signature | Description | Parameters |
|---|---|---|
#include <DataFrame/DataFrameStatsVisitors.h> template<arithmetic T, typename I = unsigned long> struct DurbinWatsonVisitor; // ------------------------------------- template<typename T, typename I = unsigned long> using dw_test_v = DurbinWatsonVisitor<T, I>; |
Durbin-Watson Test for First-Order Serial Autocorrelation in Residuals Computes the Durbin-Watson d statistic on a single column of regression residuals e:
Σi=2n (ei − eᵢ₋₁)2
The statistic ranges over [0, 4]:d ≈ 0: strong positive first-order autocorrelation d ≈ 2: no first-order autocorrelation (H0) d ≈ 4: strong negative first-order autocorrelation Note that d ≈ 2(1 − ρ̂) where ρ̂̂ is the lag-1 sample autocorrelation, so get_rho() gives ρ̂̂ without running a separate AutoCorrVisitor. Classification (get_result_category()): The Durbin-Watson bounds test classifies d against critical values dL and dU that strictly depend on n, k (regressors), and α. Because shipping the full Savin-White table inside a header is impractical, this visitor accepts user-supplied dL/dU through the constructor and falls back to the widely-used rule-of-thumb (dL=1.5, dU=2.5, α≈0.05, moderate n, k=1) when they are not provided. For precise inference look up dL and dU from the Durbin-Watson table for your n, k, and α and pass them to the constructor. The relationship between d and the bounds is: d < dL -> positive_autocorr (reject H0) d > 4−dL -> negative_autocorr (reject H0) dU < d < 4−dU -> no_autocorr (fail to reject H0) otherwise -> inconclusive
References:
Durbin, J. and Watson, G.S. (1950). "Testing for serial correlation in
least squares regression. I", Biometrika 37(3–4): 409–428.
Durbin, J. and Watson, G.S. (1951). "Testing for serial correlation in
least squares regression. II", Biometrika 38(1–2): 159–178.
Savin, N.E. and White, K.J. (1977). "The Durbin-Watson test for serial
correlation with extreme sample sizes or many regressors", Econometrica
45(8): 1989–1996.
get_result(): The Durbin-Watson d statistic ∈ [0, 4]get_rho(): Approximate lag-1 autocorrelation ρ̂ = 1 − d/2 ∈ [−1, 1] get_result_category(): Bounds-test classification of d (see dw_autocorr_t above)
explicit
DurbinWatsonVisitor(result_type d_lower = result_type(1.5),
result_type d_upper = result_type(2.5));
d_lower: Lower Durbin-Watson critical bound dL. Defaults to 1.5.d_upper: Upper Durbin-Watson critical bound dU. Defaults to 2.5. |
T: Column data type. I: Index type. |
static void test_DurbinWatsonVisitor() { std::cout << "\nTesting DurbinWatsonVisitor{ } ..." << std::endl; using MyDataFrame = StdDataFrame<unsigned long>; constexpr std::size_t col_s { 500 }; std::vector<unsigned long> idx(col_s); std::iota(idx.begin(), idx.end(), 0UL); MyDataFrame df; df.load_index(std::move(idx)); RandGenParams<double> p; p.seed = 123; // Formula // { df.load_column("x", std::vector<double>{ 1.0, -1.0, 1.0, -1.0 }, nan_policy::dont_pad_with_nans); dw_test_v<double> dw; df.single_act_visit<double>("x", dw); assert(std::abs(dw.get_result() - 3.0) < 1e-12); assert(std::abs(dw.get_rho() - -0.5) < 1e-12); // d=3 with default bounds (dL=1.5, dU=2.5): 4−dL=2.5, 4−dU=1.5 // d=3 > 4−dU=1.5 and d=3 < 4−dL=2.5? No: 3 > 2.5 = 4−dL // So d > 4−dL → negative_autocorr // assert(dw.get_result_category() == dw_autocorr_t::negative_autocorr); } // White noise // { df.load_column("y", gen_normal_dist<double>(col_s, p), nan_policy::dont_pad_with_nans); dw_test_v<double> dw; df.single_act_visit<double>("y", dw); // d should be close to 2 for white noise // assert(std::abs(dw.get_result() - 2.04668) < 1e-5); assert(std::abs(dw.get_rho() - -0.0233394) < 1e-7); assert(dw.get_result_category() == dw_autocorr_t::inconclusive); } // Positive correlation // { auto z = gen_normal_dist<double>(col_s, p); constexpr double phi { 0.9 }; for (std::size_t i = 1; i < col_s; ++i) z[i] = phi * z[i - 1] + z[i]; df.load_column("z", std::move(z)); dw_test_v<double> dw; df.single_act_visit<double>("z", dw); assert(std::abs(dw.get_result() - 0.248076) < 1e-6); assert(std::abs(dw.get_rho() - 0.875962) < 1e-6); assert(dw.get_result_category() == dw_autocorr_t::positive_autocorr); } // Negative correlation // { auto A = gen_normal_dist<double>(col_s, p); constexpr double phi { -0.9 }; for (std::size_t i = 1; i < col_s; ++i) A[i] = phi * A[i - 1] + A[i]; df.load_column<double>("A", std::move(A)); dw_test_v<double> dw; df.single_act_visit<double>("A", dw); // d should be well above 4−dL = 2.5 // assert(std::abs(dw.get_result() - 3.86027) < 1e-5); assert(std::abs(dw.get_rho() - -0.930136) < 1e-6); assert(dw.get_result_category() == dw_autocorr_t::negative_autocorr); } // Rho formula // { for (double phi : { -0.8, -0.4, 0.0, 0.4, 0.8 }) { auto B = gen_normal_dist<double>(col_s, p); for (std::size_t i = 1; i < col_s; ++i) B[i] = phi * B[i - 1] + B[i]; df.load_column("B", std::move(B)); dw_test_v<double> dw; df.single_act_visit<double>("B", dw); const double d { dw.get_result() }; const double rho { dw.get_rho() }; // The visitor formula: rho = 1 − d/2 // assert(std::abs(rho - (1.0 - d / 2.0)) < 1e-12); // For large n, sample rho ≈ phi, so d ≈ 2(1−phi) // assert(std::abs(d - 2.0 * (1.0 - phi)) < 0.3); } } // Custom bounds // { auto C = gen_normal_dist<double>(col_s, p); constexpr double phi { 0.6 }; for (std::size_t i = 1; i < col_s; ++i) C[i] = phi * C[i - 1] + C[i]; df.load_column<double>("C", std::move(C)); DurbinWatsonVisitor<double> dw_default { 1.5, 2.5 }; DurbinWatsonVisitor<double> dw_tight { 0.5, 0.6 }; df.single_act_visit<double>("C", dw_default); df.single_act_visit<double>("C", dw_tight); // d value must be identical (bounds don't affect the statistic) // assert((std::abs(dw_default.get_result() - dw_tight.get_result()) < 1e-12)); assert((std::abs(dw_default.get_rho() - dw_tight.get_rho()) < 1e-12)); // Under default bounds d<1.5 → positive_autocorr // assert((dw_default.get_result_category() == dw_autocorr_t::positive_autocorr)); // Under very tight bounds d>dU=0.6 and d<4−dU=3.4 → no_autocorr // assert(dw_tight.get_result_category() == dw_autocorr_t::no_autocorr); } // Zero residuals // { df.load_column<double>("D", std::vector<double>(col_s, 0)); DurbinWatsonVisitor<double> dw; df.single_act_visit<double>("D", dw); assert(dw.get_result() == 2.0); assert(dw.get_rho() == 0.0); assert(dw.get_result_category() == dw_autocorr_t::no_autocorr); } }