Eliminate unused virtual methods we detect after optimizations (#100286)

Before this PR, it was not possible to get rid of unused virtual method slots once whole program analysis determined they are needed. This PR makes it (sort of) possible.

Codegen needs to know what numerical slots get assigned to virtual methods because we use the slots to index into vtable to do virtual calls/generic lookup. We also want to be able to skip generating unused virtual methods. Before we do actual codegen optimizations, we don't know what virtual method uses can actually be optimized away.

### Before this PR

Without optimizations: don’t do codegen that needs slots inlined into code generated by RyuJIT. Instead instruct RyuJIT to call a helper. The helper is emitted as assembly by the compiler driver itself at the time all used virtual methods are known and we can stabilize slot numbers. When assigning slots, we skip the virtual methods that are unused.

With optimizations: assign slots after whole program analysis is done and we know what slots can be skipped. Codegen might still do optimizations that avoids actually needing a slot. We still generate the slot and fill it out and it’s a bit of a waste.

### After this PR

Without optimizations: assign slots based on what we see in metadata. Do codegen based on this information and allow inlining slot indices into codegen. Still try to find unused slots. Generate unused slots as null slots (place a literal null pointer in the vtable). This is a both an improvement and a regression. It’s debug so it doesn’t matter much. We can delete the handwritten assembly, so it’s an improvement there.

With optimizations: assign slots after whole program analysis. Still try to find unused slots. If an unused slot is found, generate it as null. We can’t eliminate it anymore for regular vtable slots (because we did codegen that assumes indices into vtable), but at least we don’t have to generate a standalone method body and a reloc to it. For sealed vtable slots, we can actually eliminate the slot (the slot is not inlined in code) and this PR does that.

src/coreclr/tools/aot/ILCompiler.Compiler/Compiler/Compilation.cs

@@ -210,9 +210,9 @@ public MethodDesc ExpandIntrinsicForCallsite(MethodDesc intrinsicMethod, MethodD
             return intrinsicMethod;
         }
 
-        public bool HasFixedSlotVTable(TypeDesc type)
+        public bool NeedsSlotUseTracking(TypeDesc type)
         {
-            return NodeFactory.VTable(type).HasFixedSlots;
+            return !NodeFactory.VTable(type).HasKnownVirtualMethodUse;
         }
 
         public bool IsEffectivelySealed(TypeDesc type)
@@ -388,43 +388,37 @@ public GenericDictionaryLookup ComputeGenericLookup(MethodDesc contextMethod, Re
                 lookupKind = ReadyToRunHelperId.TypeHandle;
             }
 
-            // Can we do a fixed lookup? Start by checking if we can get to the dictionary.
-            // Context source having a vtable with fixed slots is a prerequisite.
-            if (contextSource == GenericContextSource.MethodParameter
-                || HasFixedSlotVTable(contextMethod.OwningType))
+            DictionaryLayoutNode dictionaryLayout;
+            if (contextSource == GenericContextSource.MethodParameter)
+                dictionaryLayout = _nodeFactory.GenericDictionaryLayout(contextMethod);
+            else
+                dictionaryLayout = _nodeFactory.GenericDictionaryLayout(contextMethod.OwningType);
+
+            // If the dictionary layout has fixed slots, we can compute the lookup now. Otherwise defer to helper.
+            if (dictionaryLayout.HasFixedSlots)
             {
-                DictionaryLayoutNode dictionaryLayout;
-                if (contextSource == GenericContextSource.MethodParameter)
-                    dictionaryLayout = _nodeFactory.GenericDictionaryLayout(contextMethod);
-                else
-                    dictionaryLayout = _nodeFactory.GenericDictionaryLayout(contextMethod.OwningType);
+                int pointerSize = _nodeFactory.Target.PointerSize;
 
-                // If the dictionary layout has fixed slots, we can compute the lookup now. Otherwise defer to helper.
-                if (dictionaryLayout.HasFixedSlots)
+                GenericLookupResult lookup = ReadyToRunGenericHelperNode.GetLookupSignature(_nodeFactory, lookupKind, targetOfLookup);
+                if (dictionaryLayout.TryGetSlotForEntry(lookup, out int dictionarySlot))
                 {
-                    int pointerSize = _nodeFactory.Target.PointerSize;
+                    int dictionaryOffset = dictionarySlot * pointerSize;
 
-                    GenericLookupResult lookup = ReadyToRunGenericHelperNode.GetLookupSignature(_nodeFactory, lookupKind, targetOfLookup);
-                    if (dictionaryLayout.TryGetSlotForEntry(lookup, out int dictionarySlot))
+                    if (contextSource == GenericContextSource.MethodParameter)
                     {
-                        int dictionaryOffset = dictionarySlot * pointerSize;
-
-                        if (contextSource == GenericContextSource.MethodParameter)
-                        {
-                            return GenericDictionaryLookup.CreateFixedLookup(contextSource, dictionaryOffset);
-                        }
-                        else
-                        {
-                            int vtableSlot = VirtualMethodSlotHelper.GetGenericDictionarySlot(_nodeFactory, contextMethod.OwningType);
-                            int vtableOffset = EETypeNode.GetVTableOffset(pointerSize) + vtableSlot * pointerSize;
-                            return GenericDictionaryLookup.CreateFixedLookup(contextSource, vtableOffset, dictionaryOffset);
-                        }
+                        return GenericDictionaryLookup.CreateFixedLookup(contextSource, dictionaryOffset);
                     }
                     else
                     {
-                        return GenericDictionaryLookup.CreateNullLookup(contextSource);
+                        int vtableSlot = VirtualMethodSlotHelper.GetGenericDictionarySlot(_nodeFactory, contextMethod.OwningType);
+                        int vtableOffset = EETypeNode.GetVTableOffset(pointerSize) + vtableSlot * pointerSize;
+                        return GenericDictionaryLookup.CreateFixedLookup(contextSource, vtableOffset, dictionaryOffset);
                     }
                 }
+                else
+                {
+                    return GenericDictionaryLookup.CreateNullLookup(contextSource);
+                }
             }
 
             // Fixed lookup not possible - use helper.

src/coreclr/tools/aot/ILCompiler.Compiler/Compiler/DependencyAnalysis/EETypeNode.cs

@@ -386,7 +386,7 @@ public sealed override IEnumerable<CombinedDependencyListEntry> GetConditionalSt
             DefType defType = _type.GetClosestDefType();
 
             // If we're producing a full vtable, none of the dependencies are conditional.
-            if (!factory.VTable(defType).HasFixedSlots)
+            if (!factory.VTable(defType).HasKnownVirtualMethodUse)
             {
                 bool isNonInterfaceAbstractType = !defType.IsInterface && ((MetadataType)defType).IsAbstract;
 
@@ -1035,7 +1035,7 @@ private void OutputVirtualSlots(NodeFactory factory, ref ObjectDataBuilder objDa
 
             // It's only okay to touch the actual list of slots if we're in the final emission phase
             // or the vtable is not built lazily.
-            if (relocsOnly && !declVTable.HasFixedSlots)
+            if (relocsOnly && !declVTable.HasKnownVirtualMethodUse)
                 return;
 
             // Interface types don't place anything else in their physical vtable.
@@ -1063,13 +1063,19 @@ private void OutputVirtualSlots(NodeFactory factory, ref ObjectDataBuilder objDa
                 // No generic virtual methods can appear in the vtable!
                 Debug.Assert(!declMethod.HasInstantiation);
 
-                MethodDesc implMethod = implType.GetClosestDefType().FindVirtualFunctionTargetMethodOnObjectType(declMethod);
-
                 // Final NewSlot methods cannot be overridden, and therefore can be placed in the sealed-vtable to reduce the size of the vtable
                 // of this type and any type that inherits from it.
                 if (declMethod.CanMethodBeInSealedVTable(factory) && !declType.IsArrayTypeWithoutGenericInterfaces())
                     continue;
 
+                if (!declVTable.IsSlotUsed(declMethod))
+                {
+                    objData.EmitZeroPointer();
+                    continue;
+                }
+
+                MethodDesc implMethod = implType.GetClosestDefType().FindVirtualFunctionTargetMethodOnObjectType(declMethod);
+
                 bool shouldEmitImpl = !implMethod.IsAbstract;
 
                 // We do a size optimization that removes support for built-in ValueType Equals/GetHashCode

src/coreclr/tools/aot/ILCompiler.Compiler/Compiler/DependencyAnalysis/GenericLookupResult.cs

@@ -837,7 +837,7 @@ public override IEnumerable<DependencyNodeCore<NodeFactory>> NonRelocDependencie
 
             // If we're producing a full vtable for the type, we don't need to report virtual method use.
             // We also don't report virtual method use for generic virtual methods - tracking those is orthogonal.
-            if (!factory.VTable(canonMethod.OwningType).HasFixedSlots && !canonMethod.HasInstantiation)
+            if (!factory.VTable(canonMethod.OwningType).HasKnownVirtualMethodUse && !canonMethod.HasInstantiation)
             {
                 // Report the method as virtually used so that types that could be used here at runtime
                 // have the appropriate implementations generated.

src/coreclr/tools/aot/ILCompiler.Compiler/Compiler/DependencyAnalysis/InterfaceDispatchCellNode.cs

@@ -54,7 +54,7 @@ public override IEnumerable<DependencyListEntry> GetStaticDependencies(NodeFacto
         {
             DependencyList result = new DependencyList();
 
-            if (!factory.VTable(_targetMethod.OwningType).HasFixedSlots)
+            if (!factory.VTable(_targetMethod.OwningType).HasKnownVirtualMethodUse)
             {
                 result.Add(factory.VirtualMethodUse(_targetMethod), "Interface method use");
             }

src/coreclr/tools/aot/ILCompiler.Compiler/Compiler/DependencyAnalysis/InterfaceDispatchMapNode.cs

@@ -96,18 +96,8 @@ public static bool MightHaveInterfaceDispatchMap(TypeDesc type, NodeFactory fact
                     null :
                     (InstantiatedType)declType.GetTypeDefinition().RuntimeInterfaces[interfaceIndex];
 
-                IEnumerable<MethodDesc> slots;
-
-                // If the vtable has fixed slots, we can query it directly.
-                // If it's a lazily built vtable, we might not be able to query slots
-                // just yet, so approximate by looking at all methods.
                 VTableSliceNode vtableSlice = factory.VTable(interfaceType);
-                if (vtableSlice.HasFixedSlots)
-                    slots = vtableSlice.Slots;
-                else
-                    slots = interfaceType.GetAllVirtualMethods();
-
-                foreach (MethodDesc slotMethod in slots)
+                foreach (MethodDesc slotMethod in vtableSlice.Slots)
                 {
                     MethodDesc declMethod = slotMethod;
 
@@ -176,12 +166,16 @@ private void EmitDispatchMap(ref ObjectDataBuilder builder, NodeFactory factory)
                 if (!factory.InterfaceUse(interfaceType.GetTypeDefinition()).Marked)
                     continue;
 
-                IReadOnlyList<MethodDesc> virtualSlots = factory.VTable(interfaceType).Slots;
+                VTableSliceNode interfaceVTable = factory.VTable(interfaceType);
+                IReadOnlyList<MethodDesc> virtualSlots = interfaceVTable.Slots;
 
                 for (int interfaceMethodSlot = 0; interfaceMethodSlot < virtualSlots.Count; interfaceMethodSlot++)
                 {
                     MethodDesc declMethod = virtualSlots[interfaceMethodSlot];
 
+                    if (!interfaceVTable.IsSlotUsed(declMethod))
+                        continue;
+
                     if (!declMethod.Signature.IsStatic && !needsEntriesForInstanceInterfaceMethodImpls)
                         continue;
 

src/coreclr/tools/aot/ILCompiler.Compiler/Compiler/DependencyAnalysis/NativeLayoutVertexNode.cs

@@ -1405,7 +1405,7 @@ private static void ProcessVTableEntriesForCallingConventionSignatureGeneration(
                         break;
 
                     case VTableEntriesToProcess.AllOnTypesThatShouldProduceFullVTables:
-                        if (factory.VTable(declType).HasFixedSlots)
+                        if (factory.VTable(declType).HasKnownVirtualMethodUse)
                         {
                             vtableEntriesToProcess = factory.VTable(declType).Slots;
                         }
@@ -1416,7 +1416,7 @@ private static void ProcessVTableEntriesForCallingConventionSignatureGeneration(
                         break;
 
                     case VTableEntriesToProcess.AllOnTypesThatProducePartialVTables:
-                        if (factory.VTable(declType).HasFixedSlots)
+                        if (factory.VTable(declType).HasKnownVirtualMethodUse)
                         {
                             vtableEntriesToProcess = Array.Empty<MethodDesc>();
                         }
@@ -1639,7 +1639,7 @@ public sealed override IEnumerable<DependencyListEntry> GetStaticDependencies(No
             if (method.IsRuntimeDeterminedExactMethod)
                 method = method.GetCanonMethodTarget(CanonicalFormKind.Specific);
 
-            if (!factory.VTable(method.OwningType).HasFixedSlots)
+            if (!factory.VTable(method.OwningType).HasKnownVirtualMethodUse)
             {
                 yield return new DependencyListEntry(factory.VirtualMethodUse(method), "Slot number");
             }

src/coreclr/tools/aot/ILCompiler.Compiler/Compiler/DependencyAnalysis/NodeFactory.cs

@@ -361,7 +361,7 @@ private void CreateNodeCaches()
             {
                 // We don't need to track virtual method uses for types that have a vtable with a known layout.
                 // It's a waste of CPU time and memory.
-                Debug.Assert(!VTable(method.OwningType).HasFixedSlots);
+                Debug.Assert(method.OwningType.IsGenericDefinition || !VTable(method.OwningType).HasKnownVirtualMethodUse);
 
                 return new VariantInterfaceMethodUseNode(method);
             });
@@ -1159,7 +1159,7 @@ protected override VirtualMethodUseNode CreateValueFromKey(MethodDesc key)
             {
                 // We don't need to track virtual method uses for types that have a vtable with a known layout.
                 // It's a waste of CPU time and memory.
-                Debug.Assert(!_factory.VTable(key.OwningType).HasFixedSlots);
+                Debug.Assert(!_factory.VTable(key.OwningType).HasKnownVirtualMethodUse);
                 return new VirtualMethodUseNode(key);
             }
             protected override int GetKeyHashCode(MethodDesc key) => key.GetHashCode();

src/coreclr/tools/aot/ILCompiler.Compiler/Compiler/DependencyAnalysis/ReadyToRunGenericHelperNode.cs

@@ -154,7 +154,7 @@ public IEnumerable<DependencyListEntry> InstantiateDependencies(NodeFactory fact
                         if (createInfo.NeedsVirtualMethodUseTracking)
                         {
                             MethodDesc instantiatedTargetMethod = createInfo.TargetMethod.GetNonRuntimeDeterminedMethodFromRuntimeDeterminedMethodViaSubstitution(typeInstantiation, methodInstantiation);
-                            if (!factory.VTable(instantiatedTargetMethod.OwningType).HasFixedSlots)
+                            if (!factory.VTable(instantiatedTargetMethod.OwningType).HasKnownVirtualMethodUse)
                             {
                                 result.Add(
                                     new DependencyListEntry(

src/coreclr/tools/aot/ILCompiler.Compiler/Compiler/DependencyAnalysis/ReadyToRunHelperNode.cs

@@ -43,7 +43,7 @@ public enum ReadyToRunHelperId
         ConstrainedDirectCall,
     }
 
-    public partial class ReadyToRunHelperNode : AssemblyStubNode, INodeWithDebugInfo
+    public partial class ReadyToRunHelperNode : AssemblyStubNode
     {
         private readonly ReadyToRunHelperId _id;
         private readonly object _target;
@@ -64,7 +64,6 @@ public ReadyToRunHelperNode(ReadyToRunHelperId id, object target)
                         defType.ComputeStaticFieldLayout(StaticLayoutKind.StaticRegionSizesAndFields);
                     }
                     break;
-                case ReadyToRunHelperId.VirtualCall:
                 case ReadyToRunHelperId.ResolveVirtualFunction:
                     {
                         // Make sure we aren't trying to callvirt Object.Finalize
@@ -92,9 +91,6 @@ public override void AppendMangledName(NameMangler nameMangler, Utf8StringBuilde
         {
             switch (_id)
             {
-                case ReadyToRunHelperId.VirtualCall:
-                    sb.Append("__VirtualCall_").Append(nameMangler.GetMangledMethodName((MethodDesc)_target));
-                    break;
                 case ReadyToRunHelperId.GetNonGCStaticBase:
                     sb.Append("__GetNonGCStaticBase_").Append(nameMangler.GetMangledTypeName((TypeDesc)_target));
                     break;
@@ -122,7 +118,7 @@ public override void AppendMangledName(NameMangler nameMangler, Utf8StringBuilde
 
         protected override DependencyList ComputeNonRelocationBasedDependencies(NodeFactory factory)
         {
-            if (_id == ReadyToRunHelperId.VirtualCall || _id == ReadyToRunHelperId.ResolveVirtualFunction)
+            if (_id == ReadyToRunHelperId.ResolveVirtualFunction)
             {
                 var targetMethod = (MethodDesc)_target;
 
@@ -131,7 +127,7 @@ protected override DependencyList ComputeNonRelocationBasedDependencies(NodeFact
 #if !SUPPORT_JIT
                 factory.MetadataManager.GetDependenciesDueToVirtualMethodReflectability(ref dependencyList, factory, targetMethod);
 
-                if (!factory.VTable(targetMethod.OwningType).HasFixedSlots)
+                if (!factory.VTable(targetMethod.OwningType).HasKnownVirtualMethodUse)
 
                 {
                     dependencyList.Add(factory.VirtualMethodUse((MethodDesc)_target), "ReadyToRun Virtual Method Call");
@@ -152,7 +148,7 @@ protected override DependencyList ComputeNonRelocationBasedDependencies(NodeFact
 #if !SUPPORT_JIT
                     factory.MetadataManager.GetDependenciesDueToVirtualMethodReflectability(ref dependencyList, factory, targetMethod);
 
-                    if (!factory.VTable(info.TargetMethod.OwningType).HasFixedSlots)
+                    if (!factory.VTable(info.TargetMethod.OwningType).HasKnownVirtualMethodUse)
                     {
                         dependencyList ??= new DependencyList();
                         dependencyList.Add(factory.VirtualMethodUse(info.TargetMethod), "ReadyToRun Delegate to virtual method");
@@ -176,39 +172,6 @@ public override IEnumerable<CombinedDependencyListEntry> GetConditionalStaticDep
             return dependencyList;
         }
 
-        IEnumerable<NativeSequencePoint> INodeWithDebugInfo.GetNativeSequencePoints()
-        {
-            if (_id == ReadyToRunHelperId.VirtualCall)
-            {
-                // Generate debug information that lets debuggers step into the virtual calls.
-                // We generate a step into sequence point at the point where the helper jumps to
-                // the target of the virtual call.
-                TargetDetails target = ((MethodDesc)_target).Context.Target;
-                int debuggerStepInOffset = -1;
-                switch (target.Architecture)
-                {
-                    case TargetArchitecture.X64:
-                        debuggerStepInOffset = 3;
-                        break;
-                }
-                if (debuggerStepInOffset != -1)
-                {
-                    return new NativeSequencePoint[]
-                    {
-                        new NativeSequencePoint(0, string.Empty, WellKnownLineNumber.DebuggerStepThrough),
-                        new NativeSequencePoint(debuggerStepInOffset, string.Empty, WellKnownLineNumber.DebuggerStepIn)
-                    };
-                }
-            }
-
-            return Array.Empty<NativeSequencePoint>();
-        }
-
-        IEnumerable<DebugVarInfoMetadata> INodeWithDebugInfo.GetDebugVars()
-        {
-            return Array.Empty<DebugVarInfoMetadata>();
-        }
-
 #if !SUPPORT_JIT
         public override int ClassCode => -911637948;
 
@@ -224,7 +187,6 @@ public override int CompareToImpl(ISortableNode other, CompilerComparer comparer
                 case ReadyToRunHelperId.GetGCStaticBase:
                 case ReadyToRunHelperId.GetThreadStaticBase:
                     return comparer.Compare((TypeDesc)_target, (TypeDesc)((ReadyToRunHelperNode)other)._target);
-                case ReadyToRunHelperId.VirtualCall:
                 case ReadyToRunHelperId.ResolveVirtualFunction:
                     return comparer.Compare((MethodDesc)_target, (MethodDesc)((ReadyToRunHelperNode)other)._target);
                 case ReadyToRunHelperId.DelegateCtor:

src/coreclr/tools/aot/ILCompiler.Compiler/Compiler/DependencyAnalysis/ReflectedMethodNode.cs

@@ -68,7 +68,7 @@ public override IEnumerable<DependencyListEntry> GetStaticDependencies(NodeFacto
                     }
                     else
                     {
-                        if (ReflectionVirtualInvokeMapNode.NeedsVirtualInvokeInfo(factory, slotDefiningMethod) && !factory.VTable(slotDefiningMethod.OwningType).HasFixedSlots)
+                        if (ReflectionVirtualInvokeMapNode.NeedsVirtualInvokeInfo(factory, slotDefiningMethod) && !factory.VTable(slotDefiningMethod.OwningType).HasKnownVirtualMethodUse)
                             dependencies.Add(factory.VirtualMethodUse(slotDefiningMethod), "Virtually callable reflectable method");
                     }
                 }

src/coreclr/tools/aot/ILCompiler.Compiler/Compiler/DependencyAnalysis/ReflectionVirtualInvokeMapNode.cs

@@ -95,7 +95,7 @@ public static void GetVirtualInvokeMapDependencies(ref DependencyList dependenci
                 if (!method.HasInstantiation)
                 {
                     MethodDesc slotDefiningMethod = MetadataVirtualMethodAlgorithm.FindSlotDefiningMethodForVirtualMethod(method);
-                    if (!factory.VTable(slotDefiningMethod.OwningType).HasFixedSlots)
+                    if (!factory.VTable(slotDefiningMethod.OwningType).HasKnownVirtualMethodUse)
                     {
                         dependencies.Add(factory.VirtualMethodUse(slotDefiningMethod), "Reflection virtual invoke method");
                     }