Yarp is greatly affected by downstream services

[zhenlei520]

Describe the bug

When the response time of the downstream service of the reverse-proxy service fluctuates, the CPU fluctuation is very obvious

To Reproduce

Downstream.zip

1. Deploy Downstream on a Windows machine
2. Deploy the reverse-proxy service on the Linux server, and point the proxy address to the Downstream service on the Windows machine.
3. Use hey or other stress testing tools to stress test the reverse-proxy service according to the specified qps
4. By changing the sleep time of Downstream, check the CPU fluctuation of reverse-proxy service.

Reproduce the problem
It is found that the service where reverse-proxy is located is greatly affected by the downstream. When the downstream service responds slowly, the QPS of the reverse-proxy service will decrease, and the corresponding requests being processed will increase. When the downstream service returns to normal, the CPU of the reverse-proxy service increases rapidly. In other words, the reverse-proxy service is greatly affected by the downstream. I hope that the CPU can be more stable, even if the CPU is occupied in advance, the same principle applies to threads. I don’t want the CPU to surge or plummet when there is a burst of traffic, because the response time of the downstream service is completely uncontrollable. How to deal with this problem?

Further technical details

• Include the version of the packages you are using
• Downstream
  • deploy Windows System
  • .net version: .net 8.0
• reverse-proxy
  • deplpy Linux System
  • .net version: .net 9.0

[zhenlei520]

The reverse-proxy service only adds

ThreadPool.SetMinThreads(200, 200);
ThreadPool.SetMaxThreads(1000, 1000);

and does not do any other processing. I want to know how I can make the CPU usage of the reverse-proxy service more stable.

[MihaZupan]

Did you see a positive change after setting the min/max threads configuration / why is it necessary?

Generally, as long as you're not doing any blocking, you shouldn't need that many threads as handing a request from YARP's perspective is fully async.

Linking other threads in runtime for reference: dotnet/runtime#107017, dotnet/runtime#107013

[zhenlei520]

Did you see a positive change after setting the min/max threads configuration / why is it necessary?

Generally, as long as you're not doing any blocking, you shouldn't need that many threads as handing a request from YARP's perspective is fully async.

Linking other threads in runtime for reference: dotnet/runtime#107017, dotnet/runtime#107013

What I hope is that after setting MinThreads, when the downstream response time fluctuates and causes the reverse-proxy service to be blocked, more threads can be quickly started to help process the accumulated requests faster. But in fact, the MinThreads I set does not have much effect. I don’t know if this is related to connection reuse. And when the downstream response time becomes faster, fewer threads are blocked and fewer threads are needed to handle the problem. At this time, the threads in the thread pool will quickly decrease. I don’t know what I should do at this time to make the CPU of the reverse-proxy service more stable. The response time of the downstream service is not controllable by reverse-proxy service.

[zhenlei520]

The reason why I set MinThreads is that when reverse-proxy is blocked, it takes several minutes to create threads, and the corresponding CPU fluctuations are relatively large at this time. I hope that setting MinThreads can avoid wasting more time on thread creation and hope to process the blocked requests as soon as possible.