You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We are responsible for a large matrix of OS and architecture combinations. Testing all combinations is a waste. Maintaining images for all combinations is equally a waste.
Proposal:
Test on oldest supported version for x64
Testing on newest supported version for Arm32 and Arm64
Details
For Ubuntu, we currently support the following versions:
18.04
20.04
22.04
24.04
That's also ignoring the two or so non-LTS versions in flight at any one time.
We also support three architectures:
Arm32
Arm64
x64 (AMD64)
That's 12 combinations. However, we typically test just one Ubuntu version for each architecture to keep the matrix more sane.
The same is true for other distros we support (Alpine, Debian, RHEL family, ...).
Assumptions:
We have plenty of users wanting to run apps on old OSes. The bulk of those folks are on x64.
Most of the Arm64 usage is on cloud servers, where best-in-class performance and cost is valued, biasing to newer OS versions.
We reset Arm32 recently to require a recent OS version due to Y2038 support.
Proposal (assuming one distro version per architecture):
Use x64 to validate the min-supported version
Use Arm32 and Arm64 to validate the max-supported version
In practice, it is very uncommon to find issues "in the middle". We can rely on customer usage/reports for that.
If we opt to test more than one distro version per architecture (perhaps in outerloop), we should test the opposite of the proposal above for the given architecture.
Let's assume we only test Arm32 on Ubuntu 24.04 (for Ubuntu), then we should delete the Arm32 images for earlier Ubuntu versions and update pipelines to match.
We do have different level testing for different areas of the product.
For the runtime tests (that is JIT, GC, loader, interop), we are on the one distro per architecture plan. The exact distro versions do not matter a whole lot. If you would like to tweak the distro versions that we are run the runtime tests on, I do not see a problem with that.
For the libraries tests, we have opted to have broader distro coverage. The libraries tend to be sensitive to distro differences. For example, number of tests have conditions to account for distro differences that we consider acceptable. The adjustment of the matrix for libraries tests should be discussed with and approved by @dotnet/area-infrastructure-libraries.
Is this policy intended to apply to main as well as the servicing branches? I assume so. I can envision some nuanced limitations that may apply. For example we may choose to not test latest distro version if released in the last n months of support for .NET versions.
We are responsible for a large matrix of OS and architecture combinations. Testing all combinations is a waste. Maintaining images for all combinations is equally a waste.
Proposal:
Details
For Ubuntu, we currently support the following versions:
That's also ignoring the two or so non-LTS versions in flight at any one time.
We also support three architectures:
That's 12 combinations. However, we typically test just one Ubuntu version for each architecture to keep the matrix more sane.
The same is true for other distros we support (Alpine, Debian, RHEL family, ...).
Assumptions:
Proposal (assuming one distro version per architecture):
In practice, it is very uncommon to find issues "in the middle". We can rely on customer usage/reports for that.
If we opt to test more than one distro version per architecture (perhaps in outerloop), we should test the opposite of the proposal above for the given architecture.
Let's assume we only test Arm32 on Ubuntu 24.04 (for Ubuntu), then we should delete the Arm32 images for earlier Ubuntu versions and update pipelines to match.
Examples helix image use:
The text was updated successfully, but these errors were encountered: