Read streaming response body content

[eliasgryp2]

Hey,

I'm trying to read the response content for an Azure OpenAI response to extract some details I need to save. For most requests this works except for the streaming requests . Is there any way we can do a yield return ourselves in YARP and stream the events back to the calling server?

public static void OnResponseRecieved(TransformBuilderContext builderCtx)
{
    builderCtx.AddResponseTransform(async ctx =>
    {
        var tokenUsage = ctx.HttpContext.RequestServices.GetRequiredService<AiRequestContext>().AiTokenUsage ?? new();
        var proxyRepo = ctx.HttpContext.RequestServices.GetRequiredService<IAzureProxyRepository>();

        tokenUsage.DateEnd = DateTime.UtcNow;
        tokenUsage.ResponseType = ctx.ProxyResponse?.Content.Headers.ContentType?.ToString();

        if (!string.Equals(tokenUsage.ResponseType, "text/event-stream"))
        {
            var responseBody = await ctx.ProxyResponse?.Content.ReadAsStringAsync();

            var tokenUsageFromAzureOpenAi = JsonConvert.DeserializeObject<AzureOpenAiResponseTokenUsageDTO>(responseBody);

            tokenUsage.CompletionTokens = tokenUsageFromAzureOpenAi?.Usage.CompletionTokens ?? 0;
            tokenUsage.PromptTokens = tokenUsageFromAzureOpenAi?.Usage.PromptTokens ?? 0;
        }

        proxyRepo.SaveAiTokenUsage(tokenUsage);
    });
}