[Perf] Linux/x64: 40 Regressions on 5/27/2023 12:14:23 AM #86902

performanceautofiler · 2023-05-30T08:52:40Z

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare	71738239969b184b66e72a927a527690a761ecbf
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Tests.Perf_Basic

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
WriteBasicUtf8 - Duration of single invocation	1.87 ms	2.34 ms	1.25	0.02	False
WriteBasicUtf16 - Duration of single invocation	1.94 ms	2.39 ms	1.23	0.01	False
WriteBasicUtf8 - Duration of single invocation	520.95 ns	600.44 ns	1.15	0.01	False
WriteBasicUtf16 - Duration of single invocation	1.93 ms	2.32 ms	1.20	0.01	False
WriteBasicUtf8 - Duration of single invocation	1.83 ms	2.39 ms	1.30	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Basic*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: True, SkipValidation: True, DataSize: 100000)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.3408806339031343 > 1.92154505949183.
IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/25/2023 12:29:35 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -32.125924874135094 (T) = (0 -2311425.3056228026) / Math.Sqrt((916492552.4091985 / (38)) + (1219546242.0427265 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.2651487721284224 = (1826998.8135341406 - 2311425.3056228026) / 1826998.8135341406 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: True, SkipValidation: False, DataSize: 100000)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.3893149670329668 > 1.997571935011744.
IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/25/2023 12:29:35 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -30.551419155101406 (T) = (0 -2415472.5066380375) / Math.Sqrt((2173440929.3461156 / (38)) + (1427914196.8943276 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.27764027109606826 = (1890573.23981017 - 2415472.5066380375) / 1890573.23981017 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: True, SkipValidation: False, DataSize: 10)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 600.4378005852681 > 550.2634042503744.
IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/25/2023 12:29:35 AM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -13.555942787766599 (T) = (0 -603.4032247875114) / Math.Sqrt((729.6826798823377 / (38)) + (20.25989565113668 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.11950892195161956 = (538.9892058525174 - 603.4032247875114) / 538.9892058525174 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: True, SkipValidation: True, DataSize: 100000)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.3185141816239314 > 2.031272294717874.
IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/25/2023 12:29:35 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -29.273890304780583 (T) = (0 -2319651.048315165) / Math.Sqrt((3274466224.0751367 / (38)) + (895885445.9605526 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.2401709068259935 = (1870428.5317028742 - 2319651.048315165) / 1870428.5317028742 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: True, SkipValidation: False, DataSize: 100000)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.3887269498412698 > 1.9251214242859795.
IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/25/2023 12:29:35 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -34.45603805502824 (T) = (0 -2398405.953772479) / Math.Sqrt((1965339907.7037337 / (38)) + (1110426259.2088099 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.2838082011740852 = (1868196.4732574986 - 2398405.953772479) / 1868196.4732574986 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare	71738239969b184b66e72a927a527690a761ecbf
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.IO.Tests.BinaryWriterExtendedTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
WriteAsciiCharArray - Duration of single invocation	363.88 ns	466.47 ns	1.28	0.03	False
WriteAsciiCharArray - Duration of single invocation	105.67 μs	114.49 μs	1.08	0.03	False
WriteAsciiString - Duration of single invocation	363.79 ns	466.84 ns	1.28	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.BinaryWriterExtendedTests*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars: 8000)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 466.4734139963222 > 383.15772596179494.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/11/2023 5:02:01 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -62.654031829913194 (T) = (0 -469.92013652447446) / Math.Sqrt((26.245500407837604 / (38)) + (12.25300946969347 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.2827261375970982 = (366.3448671941504 - 469.92013652447446) / 366.3448671941504 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars: 2000000)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 114.49047945559612 > 113.30206172402872.
IsChangePoint: Marked as a change because one of 4/29/2023 8:32:21 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -13.19588940047768 (T) = (0 -115702.83254668942) / Math.Sqrt((6515911.940266402 / (38)) + (867363.3301534103 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.06850743382361006 = (108284.5368072467 - 115702.83254668942) / 108284.5368072467 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiString(StringLengthInChars: 8000)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 466.84031777918227 > 383.94951849473836.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/11/2023 5:02:01 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -54.041525304572716 (T) = (0 -468.97358170633976) / Math.Sqrt((15.48228807803085 / (38)) + (19.27088055444809 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.28077702098236795 = (366.1633321205534 - 468.97358170633976) / 366.1633321205534 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare	71738239969b184b66e72a927a527690a761ecbf
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
SerializeToStream - Duration of single invocation	5.35 μs	6.79 μs	1.27	0.01	False
SerializeObjectProperty - Duration of single invocation	6.23 μs	7.77 μs	1.25	0.01	False
SerializeToString - Duration of single invocation	5.95 μs	7.61 μs	1.28	0.01	False
SerializeToUtf8Bytes - Duration of single invocation	5.58 μs	7.15 μs	1.28	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableSortedDictionary&lt;String, String&gt;&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToStream(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.785466742077336 > 5.620330566796158.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -38.14413042760439 (T) = (0 -6792.760940070647) / Math.Sqrt((36321.29587460243 / (37)) + (1437.6788092482238 / (6))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (6) - 2, .025) and -0.24415224434961832 = (5459.750581908542 - 6792.760940070647) / 5459.750581908542 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeObjectProperty(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.770807751279954 > 6.5013251281869975.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -30.080745057111375 (T) = (0 -7658.5811372684) / Math.Sqrt((45506.34930826978 / (37)) + (5038.4359784384305 / (6))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (6) - 2, .025) and -0.21755943103309572 = (6290.108673192335 - 7658.5811372684) / 6290.108673192335 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToString(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.610254915572233 > 6.255681012242412.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -32.68505020007108 (T) = (0 -7491.1123978439255) / Math.Sqrt((32617.196150649073 / (38)) + (6809.848114679876 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.24192928079182446 = (6031.834915002383 - 7491.1123978439255) / 6031.834915002383 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToUtf8Bytes(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.151032749657221 > 5.859381495053741.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -35.90800319403818 (T) = (0 -7121.446313133833) / Math.Sqrt((29809.964933499366 / (38)) + (4944.305579525699 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.25348767610064143 = (5681.305408033432 - 7121.446313133833) / 5681.305408033432 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare	71738239969b184b66e72a927a527690a761ecbf
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Reflection.Invoke

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
StaticMethod4_int_string_struct_class - Duration of single invocation	49.76 ns	52.27 ns	1.05	0.09	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Invoke*'

Payloads

Baseline
Compare

Histogram

System.Reflection.Invoke.StaticMethod4_int_string_struct_class

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 52.266650549909954 > 50.215607555651395.
IsChangePoint: Marked as a change because one of 4/12/2023 9:47:42 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -17.12534861116317 (T) = (0 -52.382046518807556) / Math.Sqrt((1.4405795219331046 / (38)) + (0.04899866671257257 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.07547379950348426 = (48.70601826189616 - 52.382046518807556) / 48.70601826189616 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare	71738239969b184b66e72a927a527690a761ecbf
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Enum

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
InterpolateIntoSpan_NonFlags - Duration of single invocation	72.38 ns	87.61 ns	1.21	0.06	False
InterpolateIntoSpan_Flags - Duration of single invocation	75.00 ns	83.24 ns	1.11	0.04	False
InterpolateIntoSpan_Flags - Duration of single invocation	64.75 ns	71.04 ns	1.10	0.07	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Enum.InterpolateIntoSpan_NonFlags(value: 42)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 87.6100956968936 > 76.50684204353036.
IsChangePoint: Marked as a change because one of 3/20/2023 8:31:51 AM, 5/21/2023 1:46:51 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -7.481037391105198 (T) = (0 -85.35887906495647) / Math.Sqrt((11.593569699738323 / (37)) + (7.602943293148558 / (6))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (6) - 2, .025) and -0.12382509730703668 = (75.95388220951595 - 85.35887906495647) / 75.95388220951595 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.InterpolateIntoSpan_Flags(value: 32)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 83.24290799504712 > 81.17197146223633.
IsChangePoint: Marked as a change because one of 3/20/2023 8:31:51 AM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -11.57857230318279 (T) = (0 -83.14123999437209) / Math.Sqrt((10.56402685678745 / (38)) + (0.907792668164346 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.10040904835775401 = (75.55484946116331 - 83.14123999437209) / 75.55484946116331 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.InterpolateIntoSpan_Flags(value: Red)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 71.03822988869678 > 68.96792473499272.
IsChangePoint: Marked as a change because one of 3/22/2023 1:04:34 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -11.429818712205602 (T) = (0 -72.97867041955617) / Math.Sqrt((3.5585694283905367 / (38)) + (2.4303351757881004 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.1243563971602053 = (64.90706203467059 - 72.97867041955617) / 64.90706203467059 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare	71738239969b184b66e72a927a527690a761ecbf
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Uri

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
EscapeDataString - Duration of single invocation	6.16 μs	12.80 μs	2.08	0.05	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Uri*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Uri.EscapeDataString(input: "{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.800517183048434 > 6.483500356773706.
IsChangePoint: Marked as a change because one of 5/10/2023 6:05:58 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -99.0120814232714 (T) = (0 -12786.393197676012) / Math.Sqrt((177525.3632517209 / (37)) + (124.92219905417701 / (6))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (6) - 2, .025) and -1.1623430475639271 = (5913.212157562616 - 12786.393197676012) / 5913.212157562616 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

The text was updated successfully, but these errors were encountered:

performanceautofiler · 2023-05-30T08:52:44Z

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare	71738239969b184b66e72a927a527690a761ecbf
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Globalization.Tests.StringSearch

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
IndexOf_Word_NotFound - Duration of single invocation	405.85 ns	557.46 ns	1.37	0.09	False
IndexOf_Word_NotFound - Duration of single invocation	411.04 ns	557.30 ns	1.36	0.09	False
LastIndexOf_Word_NotFound - Duration of single invocation	416.49 ns	690.26 ns	1.66	0.16	False
IndexOf_Word_NotFound - Duration of single invocation	406.14 ns	557.95 ns	1.37	0.07	False
LastIndexOf_Word_NotFound - Duration of single invocation	377.18 ns	550.69 ns	1.46	0.07	False
LastIndexOf_Word_NotFound - Duration of single invocation	418.49 ns	792.76 ns	1.89	0.15	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, None, False))

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 557.4559374472741 > 425.6652644212752.
IsChangePoint: Marked as a change because one of 3/8/2023 10:34:21 AM, 4/25/2023 3:46:39 AM, 5/15/2023 9:18:21 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -125.78287533083699 (T) = (0 -557.2328841999168) / Math.Sqrt((16.217599109100632 / (38)) + (5.886258179278807 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.36580236347710837 = (407.9893981009767 - 557.2328841999168) / 407.9893981009767 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (, None, False))

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 557.3047942305857 > 425.80089114613673.
IsChangePoint: Marked as a change because one of 3/8/2023 10:34:21 AM, 4/25/2023 3:46:39 AM, 5/15/2023 9:18:21 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -144.88652499607846 (T) = (0 -557.0576792741614) / Math.Sqrt((18.416552200893843 / (38)) + (3.366451790019962 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.3623457498743924 = (408.89596442424534 - 557.0576792741614) / 408.89596442424534 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (, IgnoreCase, False))

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 690.2581159493576 > 438.0207919511483.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/15/2023 9:18:21 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -13.89956925110726 (T) = (0 -725.9599504121796) / Math.Sqrt((279.7474848662933 / (38)) + (2761.6256766343045 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.706597872862117 = (425.3843052052327 - 725.9599504121796) / 425.3843052052327 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, IgnoreNonSpace, False))

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 557.9476998326733 > 425.1856524573894.
IsChangePoint: Marked as a change because one of 3/8/2023 10:34:21 AM, 4/25/2023 3:46:39 AM, 5/15/2023 9:18:21 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -114.16025287394048 (T) = (0 -556.5581935876322) / Math.Sqrt((21.003758835223852 / (38)) + (6.7932698718628215 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.3628702329182873 = (408.37211067108353 - 556.5581935876322) / 408.37211067108353 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (, None, False))

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 550.6915677721328 > 387.2967680124046.
IsChangePoint: Marked as a change because one of 3/8/2023 10:34:21 AM, 4/25/2023 3:46:39 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -4.295362512344054 (T) = (0 -503.3537298282795) / Math.Sqrt((319.73256800566577 / (38)) + (5503.7913713100825 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.350686636382167 = (372.665069950288 - 503.3537298282795) / 372.665069950288 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, IgnoreCase, False))

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 792.755511025433 > 437.2767627338758.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/15/2023 9:18:21 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -20.228937274823224 (T) = (0 -775.7221820588065) / Math.Sqrt((272.6929682350362 / (38)) + (1733.6316487226147 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.8140331421669419 = (427.6229380970253 - 775.7221820588065) / 427.6229380970253 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare	71738239969b184b66e72a927a527690a761ecbf
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.IO.Tests.Perf_StreamWriter

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
WriteCharArray - Duration of single invocation	294.25 ms	322.09 ms	1.09	0.06	False
WriteFormat - Duration of single invocation	39.06 ns	41.24 ns	1.06	0.16	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_StreamWriter*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_StreamWriter.WriteCharArray(writeLength: 100)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 322.08740133333333 > 307.4210430625.
IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 5/5/2023 3:18:41 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -19.609912598812798 (T) = (0 -322333485.5222222) / Math.Sqrt((47720419999069.09 / (38)) + (403477287864.476 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.07524269230342202 = (299777424.975294 - 322333485.5222222) / 299777424.975294 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.Perf_StreamWriter.WriteFormat

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 41.23892068716091 > 40.294353558993606.
IsChangePoint: Marked as a change because one of 5/2/2023 3:18:41 PM, 5/26/2023 9:55:14 AM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -7.175506109206024 (T) = (0 -42.84767618404021) / Math.Sqrt((4.05025766814677 / (37)) + (2.047132336408836 / (7))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (7) - 2, .025) and -0.11877790450680994 = (38.29864355689856 - 42.84767618404021) / 38.29864355689856 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare	71738239969b184b66e72a927a527690a761ecbf
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in Benchstone.MDBenchF.MDInvMt

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Test - Duration of single invocation	2.37 ms	3.03 ms	1.28	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.MDBenchF.MDInvMt*'

Payloads

Baseline
Compare

Histogram

Benchstone.MDBenchF.MDInvMt.Test

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.033135774583333 > 2.4932704231009617.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -58.17881435490348 (T) = (0 -3035092.5423336383) / Math.Sqrt((4375129664.734865 / (38)) + (18728359.087703746 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.2633466208745493 = (2402422.654388074 - 3035092.5423336383) / 2402422.654388074 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare	71738239969b184b66e72a927a527690a761ecbf
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
TryParseUInt64Hex - Duration of single invocation	4.82 ns	5.83 ns	1.21	0.26	False
TryParseUInt64Hex - Duration of single invocation	11.29 ns	19.02 ns	1.68	0.23	False
TryParseUInt64 - Duration of single invocation	15.70 ns	21.82 ns	1.39	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64Hex(value: 3039)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.825899528304139 > 5.064194278945534.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 5/2/2023 4:25:00 AM, 5/11/2023 1:19:27 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -25.91343300307834 (T) = (0 -5.825409787829563) / Math.Sqrt((0.062952525096413 / (38)) + (7.794340251816787E-05 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.22214222377075307 = (4.766556358601257 - 5.825409787829563) / 4.766556358601257 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64Hex(value: FFFFFFFFFFFFFFFF)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.015682261431582 > 11.853379952053345.
IsChangePoint: Marked as a change because one of 4/17/2023 11:29:15 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -70.79208609612687 (T) = (0 -18.79551706896488) / Math.Sqrt((0.2086071827224402 / (38)) + (0.037371523284709066 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.6883900774475545 = (11.132212466789218 - 18.79551706896488) / 11.132212466789218 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64(value: 18446744073709551615)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.82281660426052 > 16.48157037091468.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -26.992921617292286 (T) = (0 -22.09044533223275) / Math.Sqrt((1.2367098928688385 / (38)) + (0.07538638323802585 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.3504841822623753 = (16.357426190084 - 22.09044533223275) / 16.357426190084 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare	71738239969b184b66e72a927a527690a761ecbf
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Sort<IntStruct>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
LinqQuery - Duration of single invocation	18.64 μs	21.06 μs	1.13	0.06	False
LinqOrderByExtension - Duration of single invocation	15.91 μs	21.67 μs	1.36	0.04	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;IntStruct&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Sort<IntStruct>.LinqQuery(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.06216444 > 16.708420036.
IsChangePoint: Marked as a change because one of 3/25/2023 10:49:06 AM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -3.336414956339992 (T) = (0 -22839.617866190474) / Math.Sqrt((16286447.531842625 / (38)) + (5175294.03850908 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.19902415076444238 = (19048.505279588397 - 22839.617866190474) / 19048.505279588397 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Sort<IntStruct>.LinqOrderByExtension(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.666328071428573 > 16.33941273.
IsChangePoint: Marked as a change because one of 3/25/2023 10:49:06 AM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -4.02643711215368 (T) = (0 -22852.181161111108) / Math.Sqrt((16216120.398668656 / (37)) + (3115812.0209701895 / (6))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (6) - 2, .025) and -0.20833784006054434 = (18912.07938995446 - 22852.181161111108) / 18912.07938995446 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare	71738239969b184b66e72a927a527690a761ecbf
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Guid

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Parse - Duration of single invocation	20.96 ns	25.40 ns	1.21	0.01	False
ParseExactD - Duration of single invocation	23.53 ns	26.01 ns	1.11	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Guid*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Guid.Parse

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.399690132915698 > 22.02469566640517.
IsChangePoint: Marked as a change because one of 4/1/2023 1:31:30 PM, 4/25/2023 6:46:56 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -69.43625861203486 (T) = (0 -25.41973502576306) / Math.Sqrt((0.12077176086103693 / (38)) + (0.003171535351769791 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.19948448377276554 = (21.19221663110622 - 25.41973502576306) / 21.19221663110622 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Guid.ParseExactD

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.010506410635752 > 24.510188216447286.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -31.86435954052116 (T) = (0 -25.833578849462892) / Math.Sqrt((0.038973121822451204 / (38)) + (0.028556475244863443 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.10352763741784099 = (23.409997152324365 - 25.833578849462892) / 23.409997152324365 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare	71738239969b184b66e72a927a527690a761ecbf
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Sort<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
LinqOrderByExtension - Duration of single invocation	19.65 μs	24.65 μs	1.25	0.37	False
LinqQuery - Duration of single invocation	19.60 μs	24.62 μs	1.26	0.43	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Sort<Int32>.LinqOrderByExtension(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.645369799999997 > 20.961033819500003.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -7.995370757878904 (T) = (0 -24454.909150000003) / Math.Sqrt((16178656.642191786 / (38)) + (35004.39025593646 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.2735392586936132 = (19202.320606186622 - 24454.909150000003) / 19202.320606186622 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Sort<Int32>.LinqQuery(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.624822653846152 > 20.70618416596154.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -6.009450973642971 (T) = (0 -24245.24607527473) / Math.Sqrt((15903970.829460373 / (38)) + (993261.6889140218 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.23369415218488124 = (19652.558158224405 - 24245.24607527473) / 19652.558158224405 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-05-30T08:52:45Z

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare	71738239969b184b66e72a927a527690a761ecbf
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.ReadOnlySpan

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
IndexOfString - Duration of single invocation	19.83 ns	25.89 ns	1.31	0.02	False
IndexOfString - Duration of single invocation	27.99 ns	36.97 ns	1.32	0.02	False
IndexOfString - Duration of single invocation	17.43 ns	23.04 ns	1.32	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlySpan*'

Payloads

Baseline
Compare

Histogram

System.Memory.ReadOnlySpan.IndexOfString(input: "string1", value: "string2", comparisonType: InvariantCulture)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.89154351220897 > 20.756261680525736.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -9.921972651490789 (T) = (0 -25.60540846575276) / Math.Sqrt((8.159092419223517 / (38)) + (0.052285725921656125 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.22423232890127828 = (20.91548136842053 - 25.60540846575276) / 20.91548136842053 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.ReadOnlySpan.IndexOfString(input: "foobardzsdzs", value: "rddzs", comparisonType: InvariantCulture)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 36.973547290262744 > 29.381180263733057.
IsChangePoint: Marked as a change because one of 3/8/2023 10:34:21 AM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -17.4775032902635 (T) = (0 -36.83105114392996) / Math.Sqrt((7.558831078081545 / (38)) + (0.02248832840150166 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.27165968801591933 = (28.962977666921915 - 36.83105114392996) / 28.962977666921915 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAA5AAAA", value: "5", comparisonType: InvariantCulture)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 23.042912979137256 > 18.30144091040031.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -8.794180629300222 (T) = (0 -23.228700158103738) / Math.Sqrt((8.741782720318247 / (38)) + (0.18867480004826662 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.2400758803779626 = (18.73167644468971 - 23.228700158103738) / 18.73167644468971 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare	71738239969b184b66e72a927a527690a761ecbf
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Sort<BigStruct>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
LinqOrderByExtension - Duration of single invocation	18.24 μs	31.04 μs	1.70	0.16	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;BigStruct&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Sort<BigStruct>.LinqOrderByExtension(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 31.044924885714277 > 19.395351039076925.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -5.3013711849940455 (T) = (0 -29191.9249660928) / Math.Sqrt((37703307.309724994 / (38)) + (8348933.238877036 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.3896254556291507 = (21007.045350127242 - 29191.9249660928) / 21007.045350127242 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare	71738239969b184b66e72a927a527690a761ecbf
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in Benchstone.MDBenchI.MDMulMatrix

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Test - Duration of single invocation	682.00 ms	849.36 ms	1.25	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.MDBenchI.MDMulMatrix*'

Payloads

Baseline
Compare

Histogram

Benchstone.MDBenchI.MDMulMatrix.Test

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 849.3581336 > 720.190938565.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -12.035769568749958 (T) = (0 -853158295.12558) / Math.Sqrt((4131815360925108.5 / (37)) + (149317695020701.56 / (6))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (6) - 2, .025) and -0.19739634523559418 = (712511190.2339375 - 853158295.12558) / 712511190.2339375 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

cincuranet · 2023-05-30T16:22:35Z

The diff is "always" the same: 2e45de5...b2edecd.

kunalspathak · 2023-05-30T16:32:08Z

@jakobbotsch - could it be from one of your physical promotion changes? Remind me, but it is not yet on by default, yes?

ghost · 2023-05-30T16:32:28Z

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

Issue Details

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare	71738239969b184b66e72a927a527690a761ecbf
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Tests.Perf_Basic

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
WriteBasicUtf8 - Duration of single invocation	1.87 ms	2.34 ms	1.25	0.02	False
WriteBasicUtf16 - Duration of single invocation	1.94 ms	2.39 ms	1.23	0.01	False
WriteBasicUtf8 - Duration of single invocation	520.95 ns	600.44 ns	1.15	0.01	False
WriteBasicUtf16 - Duration of single invocation	1.93 ms	2.32 ms	1.20	0.01	False
WriteBasicUtf8 - Duration of single invocation	1.83 ms	2.39 ms	1.30	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Basic*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: True, SkipValidation: True, DataSize: 100000)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.3408806339031343 > 1.92154505949183.
IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/25/2023 12:29:35 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -32.125924874135094 (T) = (0 -2311425.3056228026) / Math.Sqrt((916492552.4091985 / (38)) + (1219546242.0427265 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.2651487721284224 = (1826998.8135341406 - 2311425.3056228026) / 1826998.8135341406 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: True, SkipValidation: False, DataSize: 100000)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.3893149670329668 > 1.997571935011744.
IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/25/2023 12:29:35 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -30.551419155101406 (T) = (0 -2415472.5066380375) / Math.Sqrt((2173440929.3461156 / (38)) + (1427914196.8943276 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.27764027109606826 = (1890573.23981017 - 2415472.5066380375) / 1890573.23981017 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: True, SkipValidation: False, DataSize: 10)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 600.4378005852681 > 550.2634042503744.
IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/25/2023 12:29:35 AM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -13.555942787766599 (T) = (0 -603.4032247875114) / Math.Sqrt((729.6826798823377 / (38)) + (20.25989565113668 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.11950892195161956 = (538.9892058525174 - 603.4032247875114) / 538.9892058525174 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: True, SkipValidation: True, DataSize: 100000)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.3185141816239314 > 2.031272294717874.
IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/25/2023 12:29:35 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -29.273890304780583 (T) = (0 -2319651.048315165) / Math.Sqrt((3274466224.0751367 / (38)) + (895885445.9605526 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.2401709068259935 = (1870428.5317028742 - 2319651.048315165) / 1870428.5317028742 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: True, SkipValidation: False, DataSize: 100000)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.3887269498412698 > 1.9251214242859795.
IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/25/2023 12:29:35 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -34.45603805502824 (T) = (0 -2398405.953772479) / Math.Sqrt((1965339907.7037337 / (38)) + (1110426259.2088099 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.2838082011740852 = (1868196.4732574986 - 2398405.953772479) / 1868196.4732574986 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare	71738239969b184b66e72a927a527690a761ecbf
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.IO.Tests.BinaryWriterExtendedTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
WriteAsciiCharArray - Duration of single invocation	363.88 ns	466.47 ns	1.28	0.03	False
WriteAsciiCharArray - Duration of single invocation	105.67 μs	114.49 μs	1.08	0.03	False
WriteAsciiString - Duration of single invocation	363.79 ns	466.84 ns	1.28	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.BinaryWriterExtendedTests*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars: 8000)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 466.4734139963222 > 383.15772596179494.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/11/2023 5:02:01 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -62.654031829913194 (T) = (0 -469.92013652447446) / Math.Sqrt((26.245500407837604 / (38)) + (12.25300946969347 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.2827261375970982 = (366.3448671941504 - 469.92013652447446) / 366.3448671941504 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars: 2000000)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 114.49047945559612 > 113.30206172402872.
IsChangePoint: Marked as a change because one of 4/29/2023 8:32:21 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -13.19588940047768 (T) = (0 -115702.83254668942) / Math.Sqrt((6515911.940266402 / (38)) + (867363.3301534103 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.06850743382361006 = (108284.5368072467 - 115702.83254668942) / 108284.5368072467 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiString(StringLengthInChars: 8000)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 466.84031777918227 > 383.94951849473836.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/11/2023 5:02:01 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -54.041525304572716 (T) = (0 -468.97358170633976) / Math.Sqrt((15.48228807803085 / (38)) + (19.27088055444809 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.28077702098236795 = (366.1633321205534 - 468.97358170633976) / 366.1633321205534 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare	71738239969b184b66e72a927a527690a761ecbf
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
SerializeToStream - Duration of single invocation	5.35 μs	6.79 μs	1.27	0.01	False
SerializeObjectProperty - Duration of single invocation	6.23 μs	7.77 μs	1.25	0.01	False
SerializeToString - Duration of single invocation	5.95 μs	7.61 μs	1.28	0.01	False
SerializeToUtf8Bytes - Duration of single invocation	5.58 μs	7.15 μs	1.28	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableSortedDictionary&lt;String, String&gt;&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToStream(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.785466742077336 > 5.620330566796158.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -38.14413042760439 (T) = (0 -6792.760940070647) / Math.Sqrt((36321.29587460243 / (37)) + (1437.6788092482238 / (6))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (6) - 2, .025) and -0.24415224434961832 = (5459.750581908542 - 6792.760940070647) / 5459.750581908542 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeObjectProperty(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.770807751279954 > 6.5013251281869975.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -30.080745057111375 (T) = (0 -7658.5811372684) / Math.Sqrt((45506.34930826978 / (37)) + (5038.4359784384305 / (6))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (6) - 2, .025) and -0.21755943103309572 = (6290.108673192335 - 7658.5811372684) / 6290.108673192335 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToString(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.610254915572233 > 6.255681012242412.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -32.68505020007108 (T) = (0 -7491.1123978439255) / Math.Sqrt((32617.196150649073 / (38)) + (6809.848114679876 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.24192928079182446 = (6031.834915002383 - 7491.1123978439255) / 6031.834915002383 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToUtf8Bytes(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.151032749657221 > 5.859381495053741.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -35.90800319403818 (T) = (0 -7121.446313133833) / Math.Sqrt((29809.964933499366 / (38)) + (4944.305579525699 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.25348767610064143 = (5681.305408033432 - 7121.446313133833) / 5681.305408033432 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare	71738239969b184b66e72a927a527690a761ecbf
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Reflection.Invoke

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
StaticMethod4_int_string_struct_class - Duration of single invocation	49.76 ns	52.27 ns	1.05	0.09	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Invoke*'

Payloads

Baseline
Compare

Histogram

System.Reflection.Invoke.StaticMethod4_int_string_struct_class

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 52.266650549909954 > 50.215607555651395.
IsChangePoint: Marked as a change because one of 4/12/2023 9:47:42 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -17.12534861116317 (T) = (0 -52.382046518807556) / Math.Sqrt((1.4405795219331046 / (38)) + (0.04899866671257257 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.07547379950348426 = (48.70601826189616 - 52.382046518807556) / 48.70601826189616 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare	71738239969b184b66e72a927a527690a761ecbf
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Enum

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
InterpolateIntoSpan_NonFlags - Duration of single invocation	72.38 ns	87.61 ns	1.21	0.06	False
InterpolateIntoSpan_Flags - Duration of single invocation	75.00 ns	83.24 ns	1.11	0.04	False
InterpolateIntoSpan_Flags - Duration of single invocation	64.75 ns	71.04 ns	1.10	0.07	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Enum.InterpolateIntoSpan_NonFlags(value: 42)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 87.6100956968936 > 76.50684204353036.
IsChangePoint: Marked as a change because one of 3/20/2023 8:31:51 AM, 5/21/2023 1:46:51 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -7.481037391105198 (T) = (0 -85.35887906495647) / Math.Sqrt((11.593569699738323 / (37)) + (7.602943293148558 / (6))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (6) - 2, .025) and -0.12382509730703668 = (75.95388220951595 - 85.35887906495647) / 75.95388220951595 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.InterpolateIntoSpan_Flags(value: 32)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 83.24290799504712 > 81.17197146223633.
IsChangePoint: Marked as a change because one of 3/20/2023 8:31:51 AM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -11.57857230318279 (T) = (0 -83.14123999437209) / Math.Sqrt((10.56402685678745 / (38)) + (0.907792668164346 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.10040904835775401 = (75.55484946116331 - 83.14123999437209) / 75.55484946116331 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.InterpolateIntoSpan_Flags(value: Red)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 71.03822988869678 > 68.96792473499272.
IsChangePoint: Marked as a change because one of 3/22/2023 1:04:34 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -11.429818712205602 (T) = (0 -72.97867041955617) / Math.Sqrt((3.5585694283905367 / (38)) + (2.4303351757881004 / (6))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (6) - 2, .025) and -0.1243563971602053 = (64.90706203467059 - 72.97867041955617) / 64.90706203467059 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare	71738239969b184b66e72a927a527690a761ecbf
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Uri

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
EscapeDataString - Duration of single invocation	6.16 μs	12.80 μs	2.08	0.05	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Uri*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Uri.EscapeDataString(input: "{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.800517183048434 > 6.483500356773706.
IsChangePoint: Marked as a change because one of 5/10/2023 6:05:58 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsRegressionStdDev: Marked as regression because -99.0120814232714 (T) = (0 -12786.393197676012) / Math.Sqrt((177525.3632517209 / (37)) + (124.92219905417701 / (6))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (6) - 2, .025) and -1.1623430475639271 = (5913.212157562616 - 12786.393197676012) / 5913.212157562616 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Author:	performanceautofiler[bot]
Assignees:	-
Labels:	`os-linux`, `arch-x64`, `area-CodeGen-coreclr`, `untriaged`, `runtime-coreclr`, `needs-area-label`
Milestone:	-

cincuranet · 2023-05-30T16:39:27Z

Same here:

[Perf] Windows/x64: 20 Regressions on 5/27/2023 12:14:23 AM perf-autofiling-issues#18274

jakobbotsch · 2023-05-30T17:12:48Z

@jakobbotsch - could it be from one of your physical promotion changes? Remind me, but it is not yet on by default, yes?

Correct, not enabled yet. In theory it could be #86774, but that had quite few affected contexts in benchmarks. What about #86551 @EgorBo?

AndyAyersMS · 2023-05-30T22:16:50Z

@EgorBo's CI run reported minimal diffs but suspect this was misleading as there were quite a high number of missed contexts:

EgorBo · 2023-05-31T12:03:05Z

@EgorBo's CI run reported minimal diffs but suspect this was misleading as there were quite a high number of missed contexts:

I used a trick to avoid introducing missing contexts (see #86551 (comment)) and didn't see any diffs (as expected). I'll double check

EgorBo · 2023-05-31T13:04:26Z

Ok, I think I know what it is - it's indeed could be #86551 namely, the last minute "cosmetic" change in impMarkInlineCandidateHelper dd79675 - the destructor of InlineResult calls vm and tells it "this method was not profitable to inline - mark it as noinline for future compilations"

and I broke it. So regressions come from excessive inlining/"run out of budget", e.g. I checked Guid.Parse.

This trick we do for TP actually makes little sense since profitability heavily depends on call-site and PGO data, I already know a couple of places where we suffer from it because JIT bakes noinline too early (e.g. some non-hot callsite with non-constant args) and then even constant args don't help to inline a callee which is already has an artififcial noinline on it. I don't think it saves a lot of TP since we have other "early-out" checks like IL size, number of basic-blocks, etc. But we run into the "run out of budget" problem.

Before I "fix" that part, let's wait for tomorrow's perf reports. So far, we see more improvements than regressions from that change, e.g.:

Improvements:

Regressions:

cc @AndyAyersMS since we recently discussed the "noinline" problem in Discord

AndyAyersMS · 2023-05-31T14:39:22Z

So far, we see more improvements than regressions from that change,

I'm not sure I completely follow -- you are saying we now mark more methods as noinline, or fewer methods, or just different methods?

EgorBo · 2023-05-31T15:27:59Z

So far, we see more improvements than regressions from that change,

I'm not sure I completely follow -- you are saying we now mark more methods as noinline, or fewer methods, or just different methods?

We now mark less methods as noinline in JIT (if mark any at all) based on previous inline decisions.

EgorBo · 2023-06-01T16:36:33Z

--

EgorBo · 2023-06-10T18:01:08Z

Fixed by #87115

EgorBo · 2023-06-11T17:01:13Z

All regressions are gone with #87115 as expected

performanceautofiler bot assigned kunalspathak May 30, 2023

performanceautofiler bot added arch-x64 os-linux Linux OS (any supported distro) runtime-coreclr specific to the CoreCLR runtime untriaged New issue has not been triaged by the area owner labels May 30, 2023

cincuranet removed the untriaged New issue has not been triaged by the area owner label May 30, 2023

ghost added the untriaged New issue has not been triaged by the area owner label May 30, 2023

dotnet-issue-labeler bot added the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label May 30, 2023

cincuranet transferred this issue from dotnet/perf-autofiling-issues May 30, 2023

cincuranet unassigned kunalspathak May 30, 2023

kunalspathak added the area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI label May 30, 2023

kunalspathak mentioned this issue May 30, 2023

[Perf] Windows/x64: 72 Improvements on 5/27/2023 12:14:23 AM dotnet/perf-autofiling-issues#18341

Closed

EgorBo mentioned this issue May 31, 2023

[Perf] Windows/x64: 15 Regressions on 5/27/2023 12:14:23 AM #86905

Closed

JulieLeeMSFT assigned EgorBo Jun 2, 2023

JulieLeeMSFT removed the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label Jun 2, 2023

JulieLeeMSFT added this to the 8.0.0 milestone Jun 2, 2023

ghost removed the untriaged New issue has not been triaged by the area owner label Jun 2, 2023

EgorBo mentioned this issue Jun 4, 2023

Fix inline decision reporting + noinline propagation #87115

Merged

EgorBo closed this as completed Jun 11, 2023

ghost locked as resolved and limited conversation to collaborators Jul 11, 2023

[Perf] Linux/x64: 40 Regressions on 5/27/2023 12:14:23 AM #86902

[Perf] Linux/x64: 40 Regressions on 5/27/2023 12:14:23 AM #86902

Comments

performanceautofiler bot commented May 30, 2023

Run Information

Regressions in System.Text.Json.Tests.Perf_Basic

Repro

Payloads

Payloads

Histogram

System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: True, SkipValidation: True, DataSize: 100000)

Description of detection logic

JIT Disasms

System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: True, SkipValidation: False, DataSize: 100000)

Description of detection logic

JIT Disasms

System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: True, SkipValidation: False, DataSize: 10)

Description of detection logic

JIT Disasms

System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: True, SkipValidation: True, DataSize: 100000)

Description of detection logic

JIT Disasms

System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: True, SkipValidation: False, DataSize: 100000)

Description of detection logic

JIT Disasms

Docs

Run Information

Regressions in System.IO.Tests.BinaryWriterExtendedTests

Repro

Payloads

Payloads

Histogram

System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars: 8000)

Description of detection logic

JIT Disasms

System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars: 2000000)

Description of detection logic

JIT Disasms

System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiString(StringLengthInChars: 8000)

Description of detection logic

JIT Disasms

Docs

Run Information

Regressions in System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>

Repro

Payloads

Payloads

Histogram

System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToStream(Mode: SourceGen)

Description of detection logic

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeObjectProperty(Mode: SourceGen)

Description of detection logic

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToString(Mode: SourceGen)

Description of detection logic

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToUtf8Bytes(Mode: SourceGen)

Description of detection logic

JIT Disasms

Docs

Run Information

Regressions in System.Reflection.Invoke

Repro

Payloads

Payloads

Histogram

System.Reflection.Invoke.StaticMethod4_int_string_struct_class

Description of detection logic

JIT Disasms

Docs

Run Information

Regressions in System.Tests.Perf_Enum

Repro

Payloads

Payloads

Histogram

System.Tests.Perf_Enum.InterpolateIntoSpan_NonFlags(value: 42)

Description of detection logic

JIT Disasms