.Net: Bug: .NET Bedrock streaming completion is keeping the request thread busy

[tlecomte]

Describe the bug
The current implementation of the Bedrock connector is using response.Stream.AsEnumerable() in BedrockChatCompletionClient.cs#L193, which is keeping the current .NET thread busy while it reads the response stream. This could cause thread exhaustion issues.

To Reproduce
Steps to reproduce the behavior:

• See code at

  semantic-kernel/dotnet/src/Connectors/Connectors.Amazon/Bedrock/Core/Clients/BedrockChatCompletionClient.cs

  Line 193 in f88cf5f

  foreach (var chunk in response.Stream.AsEnumerable())

Expected behavior
Replace .AsEnumerable() with something that would read the response stream in an async manner.

Screenshots
N/A

Platform

• OS: all
• IDE: N/A
• Language: C#
• Source: NuGet package version 1.26.0

Additional context

[tlecomte]

I realize that, ideally, the AWS .NET SDK would provide a solution out of the box. So I've also created aws/aws-sdk-net#3542 and will suggest a solution there first.

[evchaki]

@RogerBarreto - can you take a look at this? @tlecomte - thanks for submitting this, we will take a look at updating this.