[Perf] Linux/x64: 7 Improvements on 4/20/2023 12:14:19 AM

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	33b18692fdf2ef438de3da38bc419c6616a746c5
Compare	7222061f18b8342a0bd4edbd37df318fd43c47d3
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Tests.Perf_Uri

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Ctor - Duration of single invocation	410.41 ns	364.89 ns	0.89	0.08	False
Ctor - Duration of single invocation	1.25 μs	1.06 μs	0.84	0.27	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Uri*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Uri.Ctor(input: "https://a.much.longer.domain.name")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 364.89463643789446 < 388.9025260008287.
IsChangePoint: Marked as a change because one of 3/2/2023 11:15:04 AM, 4/19/2023 8:38:27 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsImprovementStdDev: Marked as improvement because 16.933554056252213 (T) = (0 -369.5412789040005) / Math.Sqrt((166.54410642862229 / (299)) + (46.006277151429636 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.06827500803490213 = (396.6205501524675 - 369.5412789040005) / 396.6205501524675 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Uri.Ctor(input: "http://höst.with.ünicode")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0593218531341109 < 1.1614960796346543.
IsChangePoint: Marked as a change because one of 3/11/2023 1:35:17 AM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsImprovementStdDev: Marked as improvement because 22.1725352857483 (T) = (0 -1115.4114076652997) / Math.Sqrt((2821.474482401777 / (299)) + (1275.239985697304 / (35))) is greater than 1.9671350567190904 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (35) - 2, .975) and 0.11865814343699432 = (1265.5831552300283 - 1115.4114076652997) / 1265.5831552300283 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	33b18692fdf2ef438de3da38bc419c6616a746c5
Compare	7222061f18b8342a0bd4edbd37df318fd43c47d3
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Tests.Perf_String

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToLowerInvariant - Duration of single invocation	278.63 ns	214.46 ns	0.77	0.18	False
Concat_str_str - Duration of single invocation	530.52 ns	477.29 ns	0.90	0.03	False
ToLower - Duration of single invocation	320.30 ns	274.97 ns	0.86	0.23	False
ToUpperInvariant - Duration of single invocation	265.13 ns	222.61 ns	0.84	0.22	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.ToLowerInvariant(s: "This is a much longer piece of text that might benefit more from vectorization.")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 214.45963642370026 < 276.3277209058668.
IsChangePoint: Marked as a change because one of 3/1/2023 3:09:21 PM, 4/19/2023 8:38:27 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsImprovementStdDev: Marked as improvement because 9.044792719323247 (T) = (0 -219.43259829069942) / Math.Sqrt((2431.116231480251 / (299)) + (535.3116094325152 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.1876475979139465 = (270.119959917906 - 219.43259829069942) / 270.119959917906 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Concat_str_str(size: 1000)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 477.29353310678613 < 497.5502179047296.
IsChangePoint: Marked as a change because one of 2/14/2023 11:21:41 AM, 3/10/2023 8:06:53 PM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsImprovementStdDev: Marked as improvement because 30.175451051270922 (T) = (0 -480.50584399348145) / Math.Sqrt((952.4446921327024 / (299)) + (46.39243047034044 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.11685410607929513 = (544.0843322729921 - 480.50584399348145) / 544.0843322729921 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.ToLower(s: "This is a much longer piece of text that might benefit more from vectorization.")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 274.9684342540415 < 317.8845530456082.
IsChangePoint: Marked as a change because one of 3/1/2023 3:09:21 PM, 4/19/2023 8:38:27 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsImprovementStdDev: Marked as improvement because 8.029164269045607 (T) = (0 -272.07402373560404) / Math.Sqrt((2994.762040272043 / (299)) + (222.91770603186137 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.11697145702200322 = (308.11464238521626 - 272.07402373560404) / 308.11464238521626 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.ToUpperInvariant(s: "This is a much longer piece of text that might benefit more from vectorization.")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 222.6112856440589 < 254.213208486426.
IsChangePoint: Marked as a change because one of 3/1/2023 3:09:21 PM, 4/19/2023 8:38:27 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsImprovementStdDev: Marked as improvement because 15.159819568783325 (T) = (0 -216.05769560995213) / Math.Sqrt((2562.596638620794 / (299)) + (91.95993219266492 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.1992022751567803 = (269.8030837341002 - 216.05769560995213) / 269.8030837341002 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	33b18692fdf2ef438de3da38bc419c6616a746c5
Compare	7222061f18b8342a0bd4edbd37df318fd43c47d3
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Tests.Perf_DateTime

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToString - Duration of single invocation	487.85 ns	162.65 ns	0.33	0.13	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_DateTime*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_DateTime.ToString(format: "s")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 162.6512292283251 < 463.8037888581452.
IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/11/2023 10:53:42 PM, 4/19/2023 8:38:27 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsImprovementStdDev: Marked as improvement because 85.43025550893505 (T) = (0 -159.2572811785712) / Math.Sqrt((1937.088128457099 / (299)) + (18.08483844104 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.5911456201290549 = (389.520790333323 - 159.2572811785712) / 389.520790333323 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[lewing]

@stephentoub dotnet/runtime#84963