[Perf] Linux/x64: 2 Regressions on 5/9/2023 7:00:05 PM

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	05a3d360d6079db3cb64636fcdf99f832a58c1d2
Compare	0b5f137ab0eea82f59d22da94a0b21f0bc598f16
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Buffers.Text.Tests.Base64Tests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Base64Decode - Duration of single invocation	781.33 ns	840.93 ns	1.08	0.00	True
Base64DecodeDestinationTooSmall - Duration of single invocation	786.23 ns	843.29 ns	1.07	0.00	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Base64Tests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Base64Tests.Base64Decode(NumberOfBytes: 1000)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 840.9349449868957 > 818.4155373453012.
IsChangePoint: Marked as a change because one of 5/8/2023 3:52:55 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsRegressionStdDev: Marked as regression because -99.6258450853147 (T) = (0 -838.4983552554651) / Math.Sqrt((3.140384144129996 / (15)) + (2.1517134653950394 / (16))) is less than -2.0452296421282288 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (16) - 2, .025) and -0.07488793289483389 = (780.0797921298304 - 838.4983552554651) / 780.0797921298304 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Base64Tests.Base64DecodeDestinationTooSmall(NumberOfBytes: 1000)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 843.2885199331909 > 821.4772654526043.
IsChangePoint: Marked as a change because one of 5/8/2023 3:52:55 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsRegressionStdDev: Marked as regression because -71.08177233891763 (T) = (0 -838.2731200488503) / Math.Sqrt((6.329100373570455 / (15)) + (3.6614250867706284 / (16))) is less than -2.0452296421282288 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (16) - 2, .025) and -0.0734281313790225 = (780.9308285706367 - 838.2731200488503) / 780.9308285706367 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[radekdoulik]

range is dotnet/runtime@de33da0...b92c1d2

@stephentoub it is in base64 area, so looks like dotnet/runtime#85938. the regression is small, leaving it open in case you have an idea how to improve it, otherwise just close the issue.

[stephentoub]

This should now be fixed but I don't have close permissions.