Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/x64: 83 Improvements on 5/27/2023 12:14:23 AM #18317

Closed
performanceautofiler bot opened this issue May 30, 2023 · 5 comments
Closed

[Perf] Linux/x64: 83 Improvements on 5/27/2023 12:14:23 AM #18317

performanceautofiler bot opened this issue May 30, 2023 · 5 comments
Assignees
@kunalspathak
Labels
arch-x64 branch-refs/heads/main kind-micro os-linux perf-improvement runtime-coreclr untriaged

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented May 30, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare 71738239969b184b66e72a927a527690a761ecbf
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Tests.DictionarySequentialKeys

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryGetValue_17_Int_Int - Duration of single invocation 3.69 ns 2.23 ns 0.60 0.18 False
TryGetValue_3k_Int_32ByteValue - Duration of single invocation 3.70 ns 2.20 ns 0.59 0.08 True
TryGetValue_3k_Int_Int - Duration of single invocation 3.69 ns 2.18 ns 0.59 0.23 False
TryGetValue_3k_Int_32ByteRefsValue - Duration of single invocation 3.71 ns 2.18 ns 0.59 0.08 True

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.DictionarySequentialKeys*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.DictionarySequentialKeys.TryGetValue_17_Int_Int

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.231990217069699 < 3.504265115998343.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 154.2218504416687 (T) = (0 -2.23510552966932) / Math.Sqrt((0.025291826266862535 / (299)) + (7.907367532929845E-06 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.390068315166329 = (3.6645178226457205 - 2.23510552966932) / 3.6645178226457205 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.DictionarySequentialKeys.TryGetValue_3k_Int_32ByteValue

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.1980474591592136 < 3.5127294530551763.
IsChangePoint: Marked as a change because one of 5/19/2023 8:24:48 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 137.3778397083576 (T) = (0 -2.192362325501419) / Math.Sqrt((0.023044839289091856 / (299)) + (6.81881266061852E-05 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.37078502883808556 = (3.4842818845409567 - 2.192362325501419) / 3.4842818845409567 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.DictionarySequentialKeys.TryGetValue_3k_Int_Int

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.1819322173968647 < 3.5074751917473055.
IsChangePoint: Marked as a change because one of 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 163.07894645528043 (T) = (0 -2.179344636285916) / Math.Sqrt((0.02490804822130249 / (299)) + (1.994957784473011E-06 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.40629524463821215 = (3.670754893916727 - 2.179344636285916) / 3.670754893916727 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.DictionarySequentialKeys.TryGetValue_3k_Int_32ByteRefsValue

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.179524970362622 < 3.5162789904671277.
IsChangePoint: Marked as a change because one of 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 136.1010043276413 (T) = (0 -2.1883729304934323) / Math.Sqrt((0.02426490801751133 / (299)) + (9.126022626682889E-05 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.37908066520248 = (3.5244077738488437 - 2.1883729304934323) / 3.5244077738488437 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare 71738239969b184b66e72a927a527690a761ecbf
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Double

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToStringWithFormat - Duration of single invocation 172.43 ns 158.55 ns 0.92 0.03 False
ToStringWithFormat - Duration of single invocation 108.87 ns 96.92 ns 0.89 0.02 False
ToString - Duration of single invocation 184.67 ns 169.39 ns 0.92 0.01 False
ToStringWithCultureInfo - Duration of single invocation 101.66 ns 91.98 ns 0.90 0.01 False
ToStringWithFormat - Duration of single invocation 191.15 ns 166.60 ns 0.87 0.00 False
ToStringWithCultureInfo - Duration of single invocation 176.47 ns 165.60 ns 0.94 0.01 False
ToStringWithFormat - Duration of single invocation 176.03 ns 155.34 ns 0.88 0.04 False
ToStringWithFormat - Duration of single invocation 184.14 ns 169.82 ns 0.92 0.01 False
ToStringWithFormat - Duration of single invocation 186.96 ns 165.51 ns 0.89 0.00 False
ToString - Duration of single invocation 106.24 ns 96.94 ns 0.91 0.04 False
ToStringWithCultureInfo - Duration of single invocation 177.97 ns 163.58 ns 0.92 0.00 False
ToStringWithFormat - Duration of single invocation 128.22 ns 115.69 ns 0.90 0.03 False
ToStringWithFormat - Duration of single invocation 124.86 ns 113.69 ns 0.91 0.01 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Double*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "G17")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 158.54766348957634 < 163.63462982062399.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 16.50309982497637 (T) = (0 -158.23749973394857) / Math.Sqrt((10.824051079380236 / (299)) + (6.233532812472874 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.09758670835619694 = (175.34925648724536 - 158.23749973394857) / 175.34925648724536 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "G")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 96.92364068163849 < 103.31168433732235.
IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/25/2023 6:46:56 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 17.584201790152616 (T) = (0 -96.86504098292187) / Math.Sqrt((3.4222233736935617 / (299)) + (2.4490475942196444 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.10522037650996446 = (108.2557519639366 - 96.86504098292187) / 108.2557519639366 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToString(value: -1.7976931348623157E+308)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 169.39485564073195 < 175.32146688410856.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 34.439049460311544 (T) = (0 -169.26622419532524) / Math.Sqrt((4.7987833847587105 / (299)) + (1.8235163511632386 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.10321078176711562 = (188.74694382350282 - 169.26622419532524) / 188.74694382350282 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithCultureInfo(value: 12345, culture: zh)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 91.98087978273465 < 96.61225074942399.
IsChangePoint: Marked as a change because one of 4/9/2023 1:38:49 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 82.69542158081919 (T) = (0 -91.89100254471907) / Math.Sqrt((1.5762242581371562 / (299)) + (0.08202691263447857 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.11020935419580286 = (103.27261022357403 - 91.89100254471907) / 103.27261022357403 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "G")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 166.6043394520123 < 178.93083929461844.
IsChangePoint: Marked as a change because one of 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 15.327935014780584 (T) = (0 -169.65155638062114) / Math.Sqrt((3.9285457971347864 / (299)) + (8.322309630615878 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.09658439465724505 = (187.7890478948009 - 169.65155638062114) / 187.7890478948009 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithCultureInfo(value: 1.7976931348623157E+308, culture: zh)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 165.59802024505356 < 169.22410673041216.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 23.676240833489892 (T) = (0 -162.6821856038471) / Math.Sqrt((7.905483234992748 / (299)) + (3.9829125163608365 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.10787135080018526 = (182.3528319035188 - 162.6821856038471) / 182.3528319035188 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "G17")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 155.3422884935513 < 166.07995962496278.
IsChangePoint: Marked as a change because one of 4/17/2023 7:48:00 AM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 27.32460024516283 (T) = (0 -155.87883816701748) / Math.Sqrt((13.209879794958953 / (299)) + (3.590493037153365 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.12320631781470912 = (177.7828026526267 - 155.87883816701748) / 177.7828026526267 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "R")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 169.82280773854214 < 179.52018865555254.
IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 43.83658733658829 (T) = (0 -168.7641097330513) / Math.Sqrt((4.079793982928234 / (299)) + (1.148127782866223 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.10523104315149961 = (188.61194103946315 - 168.7641097330513) / 188.61194103946315 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "G")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 165.51301045002944 < 177.62128044128337.
IsChangePoint: Marked as a change because one of 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 68.89990543457266 (T) = (0 -165.68565032025913) / Math.Sqrt((3.667908056971264 / (299)) + (0.6326738423235156 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.12485998662842535 = (189.32473408676248 - 165.68565032025913) / 189.32473408676248 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToString(value: 12345)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 96.93596116834247 < 102.10246196464442.
IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/25/2023 6:46:56 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 12.628599037786072 (T) = (0 -98.1561847341661) / Math.Sqrt((3.9209197413144814 / (299)) + (3.5034293383770887 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.09042146555891609 = (107.91391948852545 - 98.1561847341661) / 107.91391948852545 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithCultureInfo(value: -1.7976931348623157E+308, culture: zh)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 163.58307052882517 < 171.0368099407467.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 17.78748162684293 (T) = (0 -162.80304494333038) / Math.Sqrt((7.791350443703873 / (299)) + (8.487202186577203 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.11593311910778258 = (184.15240799318983 - 162.80304494333038) / 184.15240799318983 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "E")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 115.69020377078898 < 121.19993684871034.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 25.27407515485138 (T) = (0 -116.2317567134359) / Math.Sqrt((11.596351154971947 / (299)) + (2.2841012392713846 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.12344629695584242 = (132.60083929801226 - 116.2317567134359) / 132.60083929801226 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "E")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 113.69124043797812 < 119.39267030252643.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/10/2023 6:18:38 PM, 5/11/2023 5:02:01 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 32.568725566569576 (T) = (0 -114.62773551476927) / Math.Sqrt((11.219849588169893 / (299)) + (1.2370413592627831 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.1230078593529552 = (130.70554478424052 - 114.62773551476927) / 130.70554478424052 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 30, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare 71738239969b184b66e72a927a527690a761ecbf
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 125.41 ns 115.23 ns 0.92 0.03 False
ToStringWithFormat - Duration of single invocation 141.36 ns 123.30 ns 0.87 0.01 False
ToStringWithFormat - Duration of single invocation 106.01 ns 96.99 ns 0.91 0.06 False
ToStringWithCultureInfo - Duration of single invocation 118.46 ns 107.03 ns 0.90 0.03 False
ToStringWithFormat - Duration of single invocation 127.44 ns 110.51 ns 0.87 0.03 False
ToStringWithFormat - Duration of single invocation 125.67 ns 108.98 ns 0.87 0.03 False
ToStringWithFormat - Duration of single invocation 127.34 ns 114.19 ns 0.90 0.04 False
ToStringWithFormat - Duration of single invocation 1.02 μs 915.90 ns 0.90 0.01 False
ToStringWithCultureInfo - Duration of single invocation 120.89 ns 103.63 ns 0.86 0.01 False
ToStringWithFormat - Duration of single invocation 994.78 ns 904.24 ns 0.91 0.00 False
ToString - Duration of single invocation 124.23 ns 107.78 ns 0.87 0.05 False
ToStringWithCultureInfo - Duration of single invocation 101.03 ns 91.32 ns 0.90 0.02 False
ToStringWithFormat - Duration of single invocation 137.53 ns 126.12 ns 0.92 0.02 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Single*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Single.ToString(value: -3.4028235E+38)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 115.23312095060841 < 121.71544685006809.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 18.28058288032459 (T) = (0 -115.49737404715057) / Math.Sqrt((10.597482669254592 / (299)) + (3.5230928196959943 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.11102501133868058 = (129.9219612703329 - 115.49737404715057) / 129.9219612703329 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "G17")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 123.29540594577811 < 136.29950086719737.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 50.444328803437664 (T) = (0 -124.55054061520654) / Math.Sqrt((36.22882968669588 / (299)) + (1.2271636323009711 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.18774344605984639 = (153.3389173790321 - 124.55054061520654) / 153.3389173790321 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "G")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 96.99239147073648 < 100.61610796609115.
IsChangePoint: Marked as a change because one of 4/17/2023 1:54:35 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 33.62483202237391 (T) = (0 -96.29588771689318) / Math.Sqrt((14.985415436110546 / (299)) + (1.0624628237444214 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.14268906381563676 = (112.32317663586285 - 96.29588771689318) / 112.32317663586285 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithCultureInfo(value: -3.4028235E+38, culture: zh)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 107.0327299645938 < 114.38660005779549.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 38.51598035207796 (T) = (0 -106.94255974404751) / Math.Sqrt((11.095486472369467 / (299)) + (0.8868246798799622 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.1341034191819452 = (123.50500292195825 - 106.94255974404751) / 123.50500292195825 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "R")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 110.50964595101105 < 119.72590194588722.
IsChangePoint: Marked as a change because one of 4/25/2023 8:51:41 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 26.324414988614734 (T) = (0 -110.4789865215747) / Math.Sqrt((12.468713253670044 / (299)) + (2.888727820737669 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.1470076099092979 = (129.51931084616947 - 110.4789865215747) / 129.51931084616947 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "G")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 108.98081214389643 < 118.87248330942089.
IsChangePoint: Marked as a change because one of 4/25/2023 8:51:41 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 17.181184468752775 (T) = (0 -110.28983880472303) / Math.Sqrt((11.59572891438259 / (299)) + (6.4608307328062455 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.1412912242544603 = (128.43683670167255 - 110.28983880472303) / 128.43683670167255 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "G")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 114.1894697893563 < 121.32129182217845.
IsChangePoint: Marked as a change because one of 4/25/2023 8:51:41 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 21.129533075160264 (T) = (0 -114.9594166433629) / Math.Sqrt((10.865254752487607 / (299)) + (3.0246745518774283 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.11903681520670455 = (130.49287260549528 - 114.9594166433629) / 130.49287260549528 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "F50")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 915.8975094997454 < 0.956262363739942.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 45.01855439209946 (T) = (0 -918.4594894682508) / Math.Sqrt((349.0195727206936 / (299)) + (39.60380844001039 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.12019111759757609 = (1043.930685219143 - 918.4594894682508) / 1043.930685219143 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithCultureInfo(value: 3.4028235E+38, culture: zh)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 103.63363893901064 < 114.23108008788114.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 30.092868027964663 (T) = (0 -104.05464944371818) / Math.Sqrt((10.888907781377297 / (299)) + (1.8280427862787978 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.14449707867259606 = (121.62980026095796 - 104.05464944371818) / 121.62980026095796 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "F50")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 904.236702519179 < 0.9549560665903625.
IsChangePoint: Marked as a change because one of 4/17/2023 1:30:24 AM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 40.442986847176925 (T) = (0 -906.1906209119126) / Math.Sqrt((273.49786323589166 / (299)) + (59.78929726545191 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.12831811560216239 = (1039.5886815267634 - 906.1906209119126) / 1039.5886815267634 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToString(value: 3.4028235E+38)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 107.77623275191164 < 120.3908970662852.
IsChangePoint: Marked as a change because one of 4/23/2023 2:59:55 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 14.874481432487814 (T) = (0 -110.82192148968606) / Math.Sqrt((12.70244533059891 / (299)) + (7.689079528955233 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.133779083994034 = (127.93724954215119 - 110.82192148968606) / 127.93724954215119 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithCultureInfo(value: 12345, culture: zh)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 91.3228698753718 < 96.74775701055248.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 60.122646220120245 (T) = (0 -91.60231835518262) / Math.Sqrt((10.449325229759102 / (299)) + (0.1313645576140314 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.1353088078361588 = (105.93645359790581 - 91.60231835518262) / 105.93645359790581 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "G17")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 126.12403662973321 < 132.98485490886816.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 50.15777011796873 (T) = (0 -126.67451535127566) / Math.Sqrt((40.924271429372325 / (299)) + (0.6785068560209156 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.16524834800197677 = (151.7511406513223 - 126.67451535127566) / 151.7511406513223 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare 71738239969b184b66e72a927a527690a761ecbf
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_DateTime

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 33.43 ns 31.24 ns 0.93 0.05 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_DateTime*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_DateTime.ToString(format: "o")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 31.244340877738562 < 32.07586520996607.
IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 39.68690028983956 (T) = (0 -31.24425102426534) / Math.Sqrt((2.5347904431752144 / (299)) + (0.025684744957765137 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.12547231083363164 = (35.72700031264704 - 31.24425102426534) / 35.72700031264704 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 30, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare 71738239969b184b66e72a927a527690a761ecbf
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Replace_String - Duration of single invocation 41.25 ns 34.05 ns 0.83 0.01 False
TrimEnd_CharArr - Duration of single invocation 4.50 ns 3.35 ns 0.75 0.34 False
TrimEnd_CharArr - Duration of single invocation 13.28 ns 11.75 ns 0.88 0.05 False
Replace_String - Duration of single invocation 46.21 ns 38.97 ns 0.84 0.11 False
Split - Duration of single invocation 20.40 ns 18.33 ns 0.90 0.03 False
Split - Duration of single invocation 378.76 ns 352.23 ns 0.93 0.01 False
Insert - Duration of single invocation 15.95 ns 12.51 ns 0.78 0.02 False
Join_List - Duration of single invocation 50.53 ns 41.68 ns 0.82 0.03 False
Trim_CharArr - Duration of single invocation 14.12 ns 12.85 ns 0.91 0.18 False
TrimEnd - Duration of single invocation 12.17 ns 10.24 ns 0.84 0.04 False
Concat_CharEnumerable - Duration of single invocation 6.26 μs 2.69 μs 0.43 0.03 True
Insert - Duration of single invocation 17.28 ns 13.51 ns 0.78 0.05 False
Join_Array - Duration of single invocation 47.88 ns 40.29 ns 0.84 0.03 False
Trim_CharArr - Duration of single invocation 14.23 ns 13.13 ns 0.92 0.13 False
Replace_String - Duration of single invocation 24.91 ns 20.35 ns 0.82 0.05 False
Split - Duration of single invocation 21.04 ns 18.16 ns 0.86 0.03 False
Replace_Char - Duration of single invocation 24.02 ns 21.09 ns 0.88 0.05 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence", oldValue: "nice", newValue: "bad")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 34.0469547211088 < 39.77659387326871.
IsChangePoint: Marked as a change because one of 3/9/2023 8:37:21 PM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 120.47023830036979 (T) = (0 -34.07321756426763) / Math.Sqrt((0.9564640714344019 / (299)) + (0.009845395323336767 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.19741139974728936 = (42.45415092307443 - 34.07321756426763) / 42.45415092307443 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.TrimEnd_CharArr(s: "Test", c: [' ', ' '])

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.3540783958276332 < 4.2772756222427075.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 8.6428480935218 (T) = (0 -3.427689021996135) / Math.Sqrt((0.09207347908646417 / (299)) + (0.056563479005044624 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.1992228810314708 = (4.280453250726366 - 3.427689021996135) / 4.280453250726366 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.TrimEnd_CharArr(s: "Test ", c: [' ', ' '])

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.747643916178314 < 13.132808610546624.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 15.79629681532211 (T) = (0 -11.878311559089274) / Math.Sqrt((0.20179340974361368 / (299)) + (0.10046228040044294 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.14930699497192043 = (13.963100071214523 - 11.878311559089274) / 13.963100071214523 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence. This is another very nice sentence.", oldValue: "a", newValue: "")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 38.970837055845145 < 43.76689787316857.
IsChangePoint: Marked as a change because one of 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 53.0889176678153 (T) = (0 -38.297271444193335) / Math.Sqrt((1.8115948865192961 / (299)) + (0.11568404534160844 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.1807750194940098 = (46.74817340230423 - 38.297271444193335) / 46.74817340230423 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Split(s: "ABCDEFGHIJKLMNOPQRSTUVWXYZ", arr: [' '], options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 18.32978441677769 < 19.040484793291675.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 21.032123027413316 (T) = (0 -17.4800696189194) / Math.Sqrt((0.3656371778578084 / (299)) + (0.19920400131116447 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.1824974486590322 = (21.382281425600997 - 17.4800696189194) / 21.382281425600997 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Split(s: "A B C D E F G H I J K L M N O P Q R S T U V W X Y Z", arr: [' '], options: RemoveEmptyEntries)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 352.2323800568792 < 369.50391658987036.
IsChangePoint: Marked as a change because one of 3/31/2023 8:05:26 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 37.900420626402564 (T) = (0 -350.8616814304866) / Math.Sqrt((210.89815818202547 / (299)) + (12.431126981165502 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.1525540190700821 = (414.02247379293686 - 350.8616814304866) / 414.02247379293686 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Insert(s1: "Test", i: 2, s2: " Test")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.514772186114358 < 15.283385474522976.
IsChangePoint: Marked as a change because one of 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 26.781828917825685 (T) = (0 -13.007460140804964) / Math.Sqrt((0.12161796263350375 / (299)) + (0.058464440074836976 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.17180341016297404 = (15.705763945930514 - 13.007460140804964) / 15.705763945930514 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Join_List

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 41.67781224349165 < 48.44346952290344.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/11/2023 5:02:01 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 84.69434688504386 (T) = (0 -41.741420504578606) / Math.Sqrt((5.07359709691104 / (299)) + (0.014772407037276863 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.22047548468572775 = (53.54728386925738 - 41.741420504578606) / 53.54728386925738 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Trim_CharArr(s: "Test ", c: [' ', ' '])

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.847598028451023 < 13.469585679659456.
IsChangePoint: Marked as a change because one of 4/23/2023 12:49:47 AM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 51.241494264286985 (T) = (0 -12.921573900308873) / Math.Sqrt((0.30219884082723575 / (299)) + (0.0054974092205629 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.14826661874287023 = (15.170914026214477 - 12.921573900308873) / 15.170914026214477 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.TrimEnd(s: "Test ")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 10.24105185637448 < 12.019277161503618.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 48.16103430924475 (T) = (0 -10.412003397660753) / Math.Sqrt((0.5334822876962948 / (299)) + (0.010067805593701937 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.2139402379157814 = (13.245816539512944 - 10.412003397660753) / 13.245816539512944 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Concat_CharEnumerable

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.6878208555727734 < 5.957070692259992.
IsChangePoint: Marked as a change because one of 5/22/2023 3:21:21 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 535.0799729897623 (T) = (0 -2684.416750729541) / Math.Sqrt((11792.21517126425 / (299)) + (130.90255472701477 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.5731786588742144 = (6289.3217655169565 - 2684.416750729541) / 6289.3217655169565 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Insert(s1: "dzsdzsDDZSDZSDZSddsz", i: 7, s2: "Test")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.51485712352679 < 16.364035907484453.
IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 51.78250797317894 (T) = (0 -13.6936236653516) / Math.Sqrt((0.1440346899706073 / (299)) + (0.02079367445246837 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.1919738355018001 = (16.94700526666189 - 13.6936236653516) / 16.94700526666189 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Join_Array

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 40.28520794288477 < 48.8629994745141.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/10/2023 6:18:38 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 62.29753688316997 (T) = (0 -39.859873822093746) / Math.Sqrt((3.398785822804558 / (299)) + (0.09839226909856433 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.20661986131191923 = (50.24057432040752 - 39.859873822093746) / 50.24057432040752 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Trim_CharArr(s: " Test", c: [' ', ' '])

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.126449824109072 < 13.494654302886646.
IsChangePoint: Marked as a change because one of 4/23/2023 10:00:36 AM, 5/25/2023 6:27:07 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 13.897614858157684 (T) = (0 -13.271803655264534) / Math.Sqrt((0.37775720627611337 / (299)) + (0.19961325111725103 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.13236375205461579 = (15.296506671652986 - 13.271803655264534) / 15.296506671652986 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence. This is another very nice sentence.", oldValue: "a", newValue: "b")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 20.352589802045582 < 22.74981183683068.
IsChangePoint: Marked as a change because one of 4/11/2023 3:40:00 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 32.75208098638512 (T) = (0 -19.996126403482855) / Math.Sqrt((0.39613741856595064 / (299)) + (0.10018667581992316 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.1802529074992449 = (24.39304339888761 - 19.996126403482855) / 24.39304339888761 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Split(s: "ABCDEFGHIJKLMNOPQRSTUVWXYZ", arr: [' '], options: RemoveEmptyEntries)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 18.159168096892607 < 19.261951108254134.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 23.850865422760403 (T) = (0 -17.608527278792383) / Math.Sqrt((0.2934921640027469 / (299)) + (0.15105455659676387 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.17970085493946192 = (21.465982726939053 - 17.608527278792383) / 21.465982726939053 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Replace_Char(text: "yfesgj0sg1ijslnjsb3uofdz3tbzf6ysgblu3at20nfab2wei1kxfbvsbpzwhanjczcqa2psra3aacxb67qnwbnfp2tok6v0a58l", oldChar: 'b', newChar: '+')

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 21.093246780244424 < 22.498142951936163.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 35.40592967465918 (T) = (0 -21.31970767975061) / Math.Sqrt((0.5103212385686122 / (299)) + (0.03988517509457026 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.13178784867898016 = (24.555873408719073 - 21.31970767975061) / 24.555873408719073 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare 71738239969b184b66e72a927a527690a761ecbf
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Perf_Convert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToDateTime_String - Duration of single invocation 671.80 ns 596.30 ns 0.89 0.07 False
ToBase64String - Duration of single invocation 1.17 μs 1.10 μs 0.94 0.00 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Perf_Convert*'

Payloads

Baseline
Compare

Histogram

System.Perf_Convert.ToDateTime_String(value: "Fri, 27 Feb 2009 03:11:21 GMT")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 596.2971529744817 < 623.8853277679482.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 28.52574224441702 (T) = (0 -590.6652335245034) / Math.Sqrt((457.448629543885 / (299)) + (80.03723346812048 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.15699121545287292 = (700.663201086113 - 590.6652335245034) / 700.663201086113 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Perf_Convert.ToBase64String(formattingOptions: InsertLineBreaks)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.099240879094728 < 1.114641501862989.
IsChangePoint: Marked as a change because one of 5/10/2023 6:18:38 PM, 5/16/2023 1:26:13 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 56.19535292908924 (T) = (0 -1103.238160583195) / Math.Sqrt((72.85400057632874 / (299)) + (9.62067444464322 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.06474509434518039 = (1179.6122681770503 - 1103.238160583195) / 1179.6122681770503 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare 71738239969b184b66e72a927a527690a761ecbf
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Decimal

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Divide - Duration of single invocation 51.98 ns 47.84 ns 0.92 0.02 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Decimal*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Decimal.Divide

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 47.84363203874547 < 49.84501381798107.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 80.89189912003974 (T) = (0 -47.84256214145872) / Math.Sqrt((1.5610115526557111 / (299)) + (0.004183935792459119 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.11510012078668236 = (54.06550872624268 - 47.84256214145872) / 54.06550872624268 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare 71738239969b184b66e72a927a527690a761ecbf
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_DateTimeOffset

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 40.83 ns 36.08 ns 0.88 0.02 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_DateTimeOffset*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_DateTimeOffset.ToString(format: "o")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 36.077833571885975 < 39.12291290163359.
IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 45.969144754908996 (T) = (0 -36.27620184752673) / Math.Sqrt((3.704319153257645 / (299)) + (0.032340416673106734 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.14454316526208677 = (42.40564850784165 - 36.27620184752673) / 42.40564850784165 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 30, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare 71738239969b184b66e72a927a527690a761ecbf
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.IO.Tests.Perf_Path

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetExtension - Duration of single invocation 14.88 ns 12.68 ns 0.85 0.23 False
GetFileNameWithoutExtension - Duration of single invocation 17.53 ns 16.22 ns 0.93 0.10 False
ChangeExtension - Duration of single invocation 18.75 ns 16.86 ns 0.90 0.02 False
GetFileName - Duration of single invocation 15.38 ns 14.06 ns 0.91 0.09 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_Path*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_Path.GetExtension

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.67658145770823 < 14.667553022016058.
IsChangePoint: Marked as a change because one of 4/19/2023 1:39:34 AM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 53.02586264932204 (T) = (0 -12.654296658280435) / Math.Sqrt((0.38946745957682855 / (299)) + (0.010073378197315064 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.18620085857591503 = (15.54965594598245 - 12.654296658280435) / 15.54965594598245 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.Perf_Path.GetFileNameWithoutExtension

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 16.21784817921817 < 17.20962948362825.
IsChangePoint: Marked as a change because one of 5/1/2023 6:56:14 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 15.57601830620512 (T) = (0 -15.828164155491843) / Math.Sqrt((0.6900364695893203 / (299)) + (0.19033831372075982 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.15364396391662533 = (18.701543417518213 - 15.828164155491843) / 18.701543417518213 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.Perf_Path.ChangeExtension

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 16.858901226393762 < 17.800706001190296.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 25.169319499041073 (T) = (0 -16.372912257315093) / Math.Sqrt((0.5886504806209412 / (299)) + (0.06423232100505209 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.1475309169367434 = (19.20645872397012 - 16.372912257315093) / 19.20645872397012 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.Perf_Path.GetFileName

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 14.060578460824622 < 15.24351722843852.
IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 46.50549887897143 (T) = (0 -13.892175022275126) / Math.Sqrt((0.3117478557432356 / (299)) + (0.013143632387467737 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.1599110051175286 = (16.536551611676146 - 13.892175022275126) / 16.536551611676146 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare 71738239969b184b66e72a927a527690a761ecbf
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Memory.ReadOnlySpan

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfString - Duration of single invocation 7.52 ns 5.69 ns 0.76 0.07 False
IndexOfString - Duration of single invocation 8.45 ns 6.55 ns 0.78 0.09 False
IndexOfString - Duration of single invocation 9.01 ns 7.45 ns 0.83 0.29 False
IndexOfString - Duration of single invocation 17.18 ns 15.96 ns 0.93 0.04 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlySpan*'

Payloads

Baseline
Compare

Histogram

System.Memory.ReadOnlySpan.IndexOfString(input: "Hello Worldbbbbbbbbbbbbbbcbbbbbbbbbbbbbbbbbbba!", value: "y", comparisonType: Ordinal)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.6873528071151265 < 6.864907818289515.
IsChangePoint: Marked as a change because one of 4/25/2023 9:59:52 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 68.84827860072197 (T) = (0 -5.707083337456871) / Math.Sqrt((0.1293534056230384 / (299)) + (0.0013555276532700622 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.23639536461551058 = (7.473872044508015 - 5.707083337456871) / 7.473872044508015 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.ReadOnlySpan.IndexOfString(input: "だだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだすだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだ", value: "す", comparisonType: Ordinal)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.550728757621274 < 7.924121858180914.
IsChangePoint: Marked as a change because one of 3/13/2023 2:14:21 PM, 4/25/2023 6:46:56 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 68.38728490343483 (T) = (0 -6.585496994455344) / Math.Sqrt((0.10993852921387694 / (299)) + (0.000896032784653639 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.19102071872922385 = (8.140501428059553 - 6.585496994455344) / 8.140501428059553 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "~", comparisonType: Ordinal)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.453113521867822 < 8.58260271466385.
IsChangePoint: Marked as a change because one of 3/28/2023 3:45:48 AM, 4/25/2023 3:46:39 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 10.639260347517501 (T) = (0 -7.934495314339994) / Math.Sqrt((0.2788868606779874 / (299)) + (0.13890801487561769 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.17224901762313252 = (9.585606641693468 - 7.934495314339994) / 9.585606641693468 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.ReadOnlySpan.IndexOfString(input: "More Test's", value: "Tests", comparisonType: OrdinalIgnoreCase)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 15.958205839033075 < 16.669911217612736.
IsChangePoint: Marked as a change because one of 5/2/2023 10:35:24 AM, 5/19/2023 3:32:16 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 28.700595919876278 (T) = (0 -15.853088407682337) / Math.Sqrt((1.168781051572351 / (299)) + (0.0062021400502053 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.11290827809462721 = (17.870855985029028 - 15.853088407682337) / 17.870855985029028 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare 71738239969b184b66e72a927a527690a761ecbf
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParseDouble - Duration of single invocation 22.37 ns 19.66 ns 0.88 0.01 False
TryParseUInt32 - Duration of single invocation 2.65 ns 1.43 ns 0.54 0.23 False
TryParseSingle - Duration of single invocation 23.12 ns 20.01 ns 0.87 0.02 False
TryParseUInt32Hex - Duration of single invocation 5.13 ns 3.91 ns 0.76 0.29 False
TryParseDecimal - Duration of single invocation 36.44 ns 31.86 ns 0.87 0.02 False
TryParseDateTimeOffset - Duration of single invocation 8.63 ns 6.90 ns 0.80 0.09 False

graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8ParserTests.TryParseDouble(value: 12345)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 19.657606181138366 < 21.365195157950097.
IsChangePoint: Marked as a change because one of 5/18/2023 9:42:59 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 60.58295105730195 (T) = (0 -19.52461289122105) / Math.Sqrt((0.9280174883225601 / (299)) + (0.0330838401290892 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.22363105184202142 = (25.14862674189296 - 19.52461289122105) / 25.14862674189296 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt32(value: 0)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.4299611259925487 < 2.1473715218345903.
IsChangePoint: Marked as a change because one of 4/18/2023 7:36:21 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 49.45454358594243 (T) = (0 -1.4591455537271834) / Math.Sqrt((0.038334683403911385 / (299)) + (0.0008332248789607742 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.3564565035551222 = (2.2673611990299483 - 1.4591455537271834) / 2.2673611990299483 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseSingle(value: 12345)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 20.00852412612266 < 22.000779944110487.
IsChangePoint: Marked as a change because one of 5/15/2023 2:12:16 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 27.96062018056204 (T) = (0 -19.470538288316764) / Math.Sqrt((0.5000249290262873 / (299)) + (0.18841564122762966 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.20708360367312817 = (24.555600538105445 - 19.470538288316764) / 24.555600538105445 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt32Hex(value: 3039)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.905972470635209 < 4.872652855303907.
IsChangePoint: Marked as a change because one of 3/20/2023 8:31:51 AM, 3/25/2023 6:47:18 AM, 5/22/2023 3:21:21 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 63.3647539842493 (T) = (0 -3.9059199133088938) / Math.Sqrt((0.1032477357906649 / (299)) + (9.53442926808066E-06 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.23204074943717826 = (5.086103084826863 - 3.9059199133088938) / 5.086103084826863 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseDecimal(value: 123456.789)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 31.858121842715196 < 35.4123026509052.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 5/16/2023 1:26:13 PM, 5/27/2023 7:38:18 AM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 17.65186564499108 (T) = (0 -32.71795932760372) / Math.Sqrt((2.050254940262049 / (299)) + (0.4912652895029587 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.16267595147743608 = (39.07442929094619 - 32.71795932760372) / 39.07442929094619 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseDateTimeOffset(value: 12/30/2017 3:45:22 AM -08:00)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.899438405331767 < 7.9985765765880865.
IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/25/2023 6:46:56 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 59.5632433508433 (T) = (0 -6.915890453122349) / Math.Sqrt((0.3487747694680211 / (299)) + (0.0005666353764190344 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.23420031119279883 = (9.030939231503792 - 6.915890453122349) / 9.030939231503792 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare 71738239969b184b66e72a927a527690a761ecbf
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Perf_Ascii

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToUtf16 - Duration of single invocation 4.54 ns 3.43 ns 0.76 0.16 False
FromUtf16 - Duration of single invocation 11.21 ns 9.56 ns 0.85 0.21 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Ascii*'

Payloads

Baseline
Compare

Histogram

System.Text.Perf_Ascii.ToUtf16(Size: 6)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.431181536403151 < 4.297406251269445.
IsChangePoint: Marked as a change because one of 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 39.351837226701534 (T) = (0 -3.433390681158391) / Math.Sqrt((0.07679907877908436 / (67)) + (1.8770652529421926E-05 / (6))) is greater than 1.993943367845048 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (67) + (6) - 2, .975) and 0.27983695741763565 = (4.767518573081619 - 3.433390681158391) / 4.767518573081619 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Ascii.FromUtf16(Size: 128)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 9.562743006893209 < 10.68343841413286.
IsChangePoint: Marked as a change because one of 5/26/2023 6:22:05 AM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 16.259575450257618 (T) = (0 -9.492657978425111) / Math.Sqrt((0.27346534206148093 / (63)) + (0.10594567506356457 / (8))) is greater than 1.9949454151066477 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (8) - 2, .975) and 0.18509209143388114 = (11.64874936006945 - 9.492657978425111) / 11.64874936006945 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare 71738239969b184b66e72a927a527690a761ecbf
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Enum

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
InterpolateIntoStringBuilder_Flags - Duration of single invocation 97.61 ns 88.05 ns 0.90 0.02 False
Parse_Flags - Duration of single invocation 142.36 ns 128.04 ns 0.90 0.04 False
InterpolateIntoStringBuilder_NonFlags - Duration of single invocation 92.25 ns 85.82 ns 0.93 0.03 False
TryParseGeneric_Flags - Duration of single invocation 84.97 ns 73.30 ns 0.86 0.06 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Enum.InterpolateIntoStringBuilder_Flags(value: Red, Green)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 88.05084357062005 < 91.63180845913034.
IsChangePoint: Marked as a change because one of 5/8/2023 3:52:55 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 9.944731656711294 (T) = (0 -86.5628131281946) / Math.Sqrt((11.351668783294818 / (299)) + (3.834785387090748 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.08636897777187488 = (94.74592151773572 - 86.5628131281946) / 94.74592151773572 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.Parse_Flags(text: "Red, Orange, Yellow, Green, Blue")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 128.03504889745378 < 134.58343740768413.
IsChangePoint: Marked as a change because one of 4/1/2023 8:54:51 PM, 4/25/2023 6:46:56 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 13.146595613017091 (T) = (0 -125.33351745418202) / Math.Sqrt((23.171461197432617 / (299)) + (10.353909226967694 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.12346180354032416 = (142.98694336470692 - 125.33351745418202) / 142.98694336470692 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.InterpolateIntoStringBuilder_NonFlags(value: Saturday)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 85.82493643787099 < 90.19463153527569.
IsChangePoint: Marked as a change because one of 5/8/2023 3:52:55 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 21.133076440997286 (T) = (0 -86.33620442252987) / Math.Sqrt((10.854104540613084 / (299)) + (0.16636006901289566 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.05832514404624076 = (91.68366753839437 - 86.33620442252987) / 91.68366753839437 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.TryParseGeneric_Flags(text: "Red, Orange, Yellow, Green, Blue")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 73.30179404200223 < 80.74857054391596.
IsChangePoint: Marked as a change because one of 4/1/2023 8:54:51 PM, 4/17/2023 7:48:00 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 11.083243510577027 (T) = (0 -76.20248649267198) / Math.Sqrt((19.83213647820658 / (299)) + (2.083877853812557 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.0855409168383291 = (83.33066825604469 - 76.20248649267198) / 83.33066825604469 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare 71738239969b184b66e72a927a527690a761ecbf
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Tests.Perf_StringBuilder

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Insert_Primitives - Duration of single invocation 22.69 μs 18.54 μs 0.82 0.00 True
Insert_Strings - Duration of single invocation 428.09 ns 383.02 ns 0.89 0.01 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_StringBuilder*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_StringBuilder.Insert_Primitives

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 18.54431446080673 < 21.661127250320632.
IsChangePoint: Marked as a change because one of 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 95.02423460788542 (T) = (0 -18643.617232323737) / Math.Sqrt((53561.11365050587 / (299)) + (13385.127600562066 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.2001369752508903 = (23308.512402072363 - 18643.617232323737) / 23308.512402072363 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Tests.Perf_StringBuilder.Insert_Strings

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 383.0194151005073 < 407.60600369700916.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 18.769515996491993 (T) = (0 -374.48905067675963) / Math.Sqrt((177.72531001497472 / (299)) + (85.79117748895115 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.16207284893474863 = (446.92316056434515 - 374.48905067675963) / 446.92316056434515 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 30, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare 71738239969b184b66e72a927a527690a761ecbf
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Concurrent.IsEmpty<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Queue - Duration of single invocation 5.28 ns 3.85 ns 0.73 0.21 False
Queue - Duration of single invocation 5.13 ns 4.09 ns 0.80 0.22 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.IsEmpty&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Concurrent.IsEmpty<String>.Queue(Size: 0)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.851249203736666 < 5.011589367082053.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 4/8/2023 3:09:32 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 66.87268452703174 (T) = (0 -3.8414078607420072) / Math.Sqrt((0.12747566898181473 / (299)) + (0.00015029819133994204 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.2699969453279764 = (5.262180529460768 - 3.8414078607420072) / 5.262180529460768 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Concurrent.IsEmpty<String>.Queue(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.0910753414133145 < 4.876504872296542.
IsChangePoint: Marked as a change because one of 4/7/2023 2:04:00 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 52.994538475099596 (T) = (0 -4.0968763382943445) / Math.Sqrt((0.13810711085093894 / (299)) + (5.736823341466914E-05 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.21927845309076469 = (5.247551261411051 - 4.0968763382943445) / 5.247551261411051 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare 71738239969b184b66e72a927a527690a761ecbf
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Globalization.Tests.Perf_NumberCultureInfo

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 102.23 ns 94.38 ns 0.92 0.00 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.Perf_NumberCultureInfo*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: ja)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 94.38241439286777 < 98.00927338917793.
IsChangePoint: Marked as a change because one of 4/8/2023 8:10:53 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 37.840735070237876 (T) = (0 -95.07071174668862) / Math.Sqrt((5.317437405031192 / (299)) + (0.37216167457381094 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.10108001090950305 = (105.76103869141747 - 95.07071174668862) / 105.76103869141747 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare 71738239969b184b66e72a927a527690a761ecbf
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in PerfLabTests.BlockCopyPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CallBlockCopy - Duration of single invocation 6.36 ns 4.88 ns 0.77 0.15 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.BlockCopyPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.BlockCopyPerf.CallBlockCopy(numElements: 100)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.881963029108116 < 6.03998528384182.
IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 5/2/2023 10:35:24 AM, 5/8/2023 3:52:55 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 14.720372691748457 (T) = (0 -4.967874311649775) / Math.Sqrt((0.5030914493011461 / (299)) + (0.029694056441667355 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.1943928676368561 = (6.166621560408932 - 4.967874311649775) / 6.166621560408932 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare 71738239969b184b66e72a927a527690a761ecbf
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
HeapSort - Duration of single invocation 53.90 μs 50.30 μs 0.93 0.08 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_PriorityQueue&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.HeapSort(Size: 1000)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 50.300017654914534 < 51.31453676686851.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 55.58469940565623 (T) = (0 -50316.13744087313) / Math.Sqrt((1830764.710941934 / (299)) + (11591.753519243337 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.09020338593061686 = (55304.819409930125 - 50316.13744087313) / 55304.819409930125 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare 71738239969b184b66e72a927a527690a761ecbf
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Globalization.Tests.Perf_DateTimeCultureInfo

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToStringHebrewIsrael - Duration of single invocation 230.46 ns 187.13 ns 0.81 0.01 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.Perf_DateTimeCultureInfo*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.Perf_DateTimeCultureInfo.ToStringHebrewIsrael

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 187.12801560676093 < 219.56522027965607.
IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 38.87014512923378 (T) = (0 -187.19923710336968) / Math.Sqrt((295.93696493259586 / (299)) + (13.507253525365844 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.27209653980841136 = (257.17591320983377 - 187.19923710336968) / 257.17591320983377 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare 71738239969b184b66e72a927a527690a761ecbf
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Tests.Perf_Booleans

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteBooleans - Duration of single invocation 1.41 ms 1.20 ms 0.85 0.01 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Booleans*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Booleans.WriteBooleans(Formatted: True, SkipValidation: True)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1962528966836736 < 1.3416351273968474.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 25.77649829172923 (T) = (0 -1220574.221830135) / Math.Sqrt((1536063898.9172761 / (299)) + (769178343.8763182 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.19604707095538831 = (1518216.0270012585 - 1220574.221830135) / 1518216.0270012585 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare 71738239969b184b66e72a927a527690a761ecbf
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in BenchmarksGame.KNucleotide_1

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
RunBench - Duration of single invocation 236.39 ms 196.00 ms 0.83 0.03 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'BenchmarksGame.KNucleotide_1*'

Payloads

Baseline
Compare

Histogram

BenchmarksGame.KNucleotide_1.RunBench

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 196.00209405555555 < 218.05422411261904.
IsChangePoint: Marked as a change because one of 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 84.83342757022847 (T) = (0 -193625742.72913083) / Math.Sqrt((12683527795255.924 / (299)) + (1478531031310.82 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.19059081428747493 = (239218612.96728623 - 193625742.72913083) / 239218612.96728623 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 7b2dc26555db2ef75c4ef822613a63b8c1d25e85
Compare 71738239969b184b66e72a927a527690a761ecbf
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Tests.Perf_Strings

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteStringsUtf16 - Duration of single invocation 10.26 ms 9.35 ms 0.91 0.02 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Strings*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: False, SkipValidation: True, Escaped: OneEscaped)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 9.3452815375 < 9.7600492.
IsChangePoint: Marked as a change because one of 5/4/2023 10:13:35 PM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM falls between 5/21/2023 1:46:51 AM and 5/29/2023 1:48:21 PM.
IsImprovementStdDev: Marked as improvement because 8.662497180127023 (T) = (0 -9423076.396130953) / Math.Sqrt((94980975782.8746 / (299)) + (75938221766.71896 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.09478500217894496 = (10409766.09845535 - 9423076.396130953) / 10409766.09845535 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@kunalspathak
Copy link
Member

dotnet/runtime#86902

@EgorBo EgorBo mentioned this issue May 31, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kunalspathak kunalspathak

Labels
arch-x64 branch-refs/heads/main kind-micro os-linux perf-improvement runtime-coreclr untriaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@kunalspathak