Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect Values for MovingStatistics StandardDeviation #1100

Open
thoblerone opened this issue Oct 22, 2024 · 0 comments
Open

Incorrect Values for MovingStatistics StandardDeviation #1100

thoblerone opened this issue Oct 22, 2024 · 0 comments

Comments

@thoblerone
Copy link

I created a simple test case to get comfortable with the MovingStatistics class and surprisingly stubled into a calculation error:

public void TryingOutMathNetStatisticsOptions()
{
    var dataList = new List<double>()
    {
        1, 1, 1, 1, 1,
        2, 2, 2, 2, 2,
        3, 3, 3, 3, 3,
        4, 4, 4, 4, 4,
        3, 3, 3, 3, 3,
        2, 2, 2, 2, 2,
        1, 1, 1, 1, 1
    };

    var statistics = new MathNet.Numerics.Statistics.MovingStatistics(5);
    var msCount = new List<double>();
    var msMean = new List<double>();
    var msStdDev = new List<double>();
    var msPopStdDev = new List<double>();

    foreach (var data in dataList)
    {
        statistics.Push(data);

        msCount.Add(statistics.Count);
        msMean.Add(statistics.Mean);
        msStdDev.Add(statistics.StandardDeviation);
        msPopStdDev.Add(statistics.PopulationStandardDeviation);

        Trace.WriteLine($"{data}\t{statistics.Count}\t{statistics.Mean}\t"+
                        $"{statistics.StandardDeviation}\t{statistics.PopulationStandardDeviation}");
    }

    for (int i = 4; i < dataList.Count; i += 5)
    {
        Assert.That(msStdDev[i], Is.Not.NaN);
        Assert.That(msPopStdDev[i], Is.Not.NaN);
    }
}

The data set was designed to be used with a window width of 5 with 5 consecutive identical values followed by the next set of 5. As both standard deviations step down to zero, for the later data the calculation behind the statistics.Push(data) call will accumulate a small rounding error in the magnitude of 1E-15 that will finally yield in a negative variance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant