Back to Documentations

Signature Description
enum class  dw_autocorr_t : unsigned char  {

    positive_autocorr = 1,  // d < dL: reject H0 in favour of +ve AR(1)
    negative_autocorr = 2,  // d > 4−dL: reject H0 in favour of −ve AR(1)
    no_autocorr = 3,        // dU < d < 4−dU: fail to reject H0
    inconclusive = 4,       // dL <= d <= dU or 4−dU <= d <= 4−dL
};
Interpretation of the Durbin-Watson d statistic.
The Durbin-Watson bounds test compares d against critical values dL and dU that depend on n (sample size), k (number of regressors excluding the intercept), and the chosen significance level. Rather than shipping the full table of critical values inside the visitor, DurbinWatsonVisitor returns one of these four classifications based on the commonly-used rule-of-thumb thresholds (see get_result_category() documentation).

Signature Description Parameters
#include <DataFrame/DataFrameStatsVisitors.h>

template<arithmetic T, typename I = unsigned long>
struct DurbinWatsonVisitor;

// -------------------------------------

template<typename T, typename I = unsigned long>
using dw_test_v = DurbinWatsonVisitor<T, I>;
Durbin-Watson Test for First-Order Serial Autocorrelation in Residuals
Computes the Durbin-Watson d statistic on a single column of regression residuals e:
      Σi=2n (ei − eᵢ₋₁)2
d = ────────────────────--
Σi=1n ei2
The statistic ranges over [0, 4]:
d ≈ 0: strong positive first-order autocorrelation
d ≈ 2: no first-order autocorrelation (H0)
d ≈ 4: strong negative first-order autocorrelation

Note that d ≈ 2(1 − ρ̂) where ρ̂̂ is the lag-1 sample autocorrelation, so get_rho() gives ρ̂̂ without running a separate AutoCorrVisitor.

Classification (get_result_category()): The Durbin-Watson bounds test classifies d against critical values dL and dU that strictly depend on n, k (regressors), and α. Because shipping the full Savin-White table inside a header is impractical, this visitor accepts user-supplied dL/dU through the constructor and falls back to the widely-used rule-of-thumb (dL=1.5, dU=2.5, α≈0.05, moderate n, k=1) when they are not provided. For precise inference look up dL and dU from the Durbin-Watson table for your n, k, and α and pass them to the constructor.

The relationship between d and the bounds is:
d < dL -> positive_autocorr (reject H0)
d > 4−dL -> negative_autocorr (reject H0)
dU < d < 4−dU -> no_autocorr (fail to reject H0)
otherwise -> inconclusive

References:
  Durbin, J. and Watson, G.S. (1950). "Testing for serial correlation in
  least squares regression. I", Biometrika 37(3–4): 409–428.

  Durbin, J. and Watson, G.S. (1951). "Testing for serial correlation in
  least squares regression. II", Biometrika 38(1–2): 159–178.

  Savin, N.E. and White, K.J. (1977). "The Durbin-Watson test for serial
  correlation with extreme sample sizes or many regressors", Econometrica
  45(8): 1989–1996.
        
get_result(): The Durbin-Watson d statistic ∈ [0, 4]
get_rho(): Approximate lag-1 autocorrelation ρ̂ = 1 − d/2 ∈ [−1, 1]
get_result_category(): Bounds-test classification of d (see dw_autocorr_t above)
    explicit
    DurbinWatsonVisitor(result_type d_lower = result_type(1.5),
                        result_type d_upper = result_type(2.5));
        
d_lower: Lower Durbin-Watson critical bound dL. Defaults to 1.5.
d_upper: Upper Durbin-Watson critical bound dU. Defaults to 2.5.
T: Column data type.
I: Index type.
static void test_DurbinWatsonVisitor()  {

    std::cout << "\nTesting DurbinWatsonVisitor{ } ..." << std::endl;

    using MyDataFrame = StdDataFrame<unsigned long>;

    constexpr std::size_t   col_s { 500 };

    std::vector<unsigned long>  idx(col_s);

    std::iota(idx.begin(), idx.end(), 0UL);

    MyDataFrame df;

    df.load_index(std::move(idx));

    RandGenParams<double>   p;

    p.seed = 123;

    // Formula
    //
    {
        df.load_column("x", std::vector<double>{ 1.0, -1.0, 1.0, -1.0 }, nan_policy::dont_pad_with_nans);

        dw_test_v<double>   dw;

        df.single_act_visit<double>("x", dw);

        assert(std::abs(dw.get_result() - 3.0) < 1e-12);
        assert(std::abs(dw.get_rho() - -0.5) < 1e-12);

        // d=3 with default bounds (dL=1.5, dU=2.5): 4−dL=2.5, 4−dU=1.5
        // d=3 > 4−dU=1.5 and d=3 < 4−dL=2.5? No: 3 > 2.5 = 4−dL
        // So d > 4−dL → negative_autocorr
        //
        assert(dw.get_result_category() == dw_autocorr_t::negative_autocorr);
    }

    // White noise
    //
    {
        df.load_column("y", gen_normal_dist<double>(col_s, p), nan_policy::dont_pad_with_nans);

        dw_test_v<double>   dw;

        df.single_act_visit<double>("y", dw);

        // d should be close to 2 for white noise
        //
        assert(std::abs(dw.get_result() - 2.04668) < 1e-5);
        assert(std::abs(dw.get_rho() - -0.0233394) < 1e-7);
        assert(dw.get_result_category() == dw_autocorr_t::inconclusive);
    }

    // Positive correlation
    //
    {
        auto                z = gen_normal_dist<double>(col_s, p);
        constexpr double    phi { 0.9 };

        for (std::size_t i = 1; i < col_s; ++i)
            z[i] = phi * z[i - 1] + z[i];

        df.load_column("z", std::move(z));

        dw_test_v<double>   dw;

        df.single_act_visit<double>("z", dw);

        assert(std::abs(dw.get_result() - 0.248076) < 1e-6);
        assert(std::abs(dw.get_rho() - 0.875962) < 1e-6);
        assert(dw.get_result_category() == dw_autocorr_t::positive_autocorr);
    }

    // Negative correlation
    //
    {
        auto                A = gen_normal_dist<double>(col_s, p);
        constexpr double    phi { -0.9 };

        for (std::size_t i = 1; i < col_s; ++i)
            A[i] = phi * A[i - 1] + A[i];

        df.load_column<double>("A", std::move(A));

        dw_test_v<double>   dw;

        df.single_act_visit<double>("A", dw);

        // d should be well above 4−dL = 2.5
        //
        assert(std::abs(dw.get_result() - 3.86027) < 1e-5);
        assert(std::abs(dw.get_rho() - -0.930136) < 1e-6);
        assert(dw.get_result_category() == dw_autocorr_t::negative_autocorr);
    }

    // Rho formula
    //
    {
        for (double phi : { -0.8, -0.4, 0.0, 0.4, 0.8 })  {
            auto    B = gen_normal_dist<double>(col_s, p);

            for (std::size_t i = 1; i < col_s; ++i)
                B[i] = phi * B[i - 1] + B[i];

            df.load_column("B", std::move(B));

            dw_test_v<double>   dw;

            df.single_act_visit<double>("B", dw);

            const double    d { dw.get_result() };
            const double    rho { dw.get_rho() };

            // The visitor formula: rho = 1 − d/2
            //
            assert(std::abs(rho - (1.0 - d / 2.0)) < 1e-12);

            // For large n, sample rho ≈ phi, so d ≈ 2(1−phi)
            //
            assert(std::abs(d - 2.0 * (1.0 - phi)) < 0.3);
        }
    }

    // Custom bounds
    //
    {
        auto                C = gen_normal_dist<double>(col_s, p);
        constexpr double    phi { 0.6 };

        for (std::size_t i = 1; i < col_s; ++i)
            C[i] = phi * C[i - 1] + C[i];

        df.load_column<double>("C", std::move(C));

        DurbinWatsonVisitor<double> dw_default { 1.5, 2.5 };
        DurbinWatsonVisitor<double> dw_tight { 0.5, 0.6 };

        df.single_act_visit<double>("C", dw_default);
        df.single_act_visit<double>("C", dw_tight);

        // d value must be identical (bounds don't affect the statistic)
        //
        assert((std::abs(dw_default.get_result() - dw_tight.get_result()) < 1e-12));
        assert((std::abs(dw_default.get_rho() - dw_tight.get_rho()) < 1e-12));

        // Under default bounds d<1.5 → positive_autocorr
        //
        assert((dw_default.get_result_category() == dw_autocorr_t::positive_autocorr));

        // Under very tight bounds d>dU=0.6 and d<4−dU=3.4 → no_autocorr
        //
        assert(dw_tight.get_result_category() == dw_autocorr_t::no_autocorr);
    }

    // Zero residuals
    //
    {
        df.load_column<double>("D", std::vector<double>(col_s, 0));

        DurbinWatsonVisitor<double> dw;

        df.single_act_visit<double>("D", dw);

        assert(dw.get_result() == 2.0);
        assert(dw.get_rho() == 0.0);
        assert(dw.get_result_category() == dw_autocorr_t::no_autocorr);
    }
}

C++ DataFrame