Design Doc: AWS Messaging Framework

[ashovlin]

The AWS .NET team is exploring creating an AWS native framework that simplifies development of .NET message processing applications using AWS services.

The design doc can be viewed and commented on in PR 41 (rendered view).

The purpose of the framework would be to reduce the amount of boiler-plate code developers need to write. The primary responsibilities of the proposed framework are:

• Handling the message routing - In a publisher, the framework will handle routing the messages to the correct queue/topic/eventbus. In a consumer process, it will route the particular message type to the appropriate business logic.
• Handling the overall message lifecycle - The framework will handle serializing/deserializing the message to .NET objects, keeping track of the message visibility while it is being processed, and deleting the message when completed.

Here is an example showing a sample publisher and handler for a hypothetical OrderInfo message.

Sample publisher:

[ApiController]
[Route("[controller]")]
public class OrderController : ControllerBase
{
    // See later in the design for how this was configured and mapped to the queue
    private readonly IMessagePublisher _publisher;

    public OrderController(IMessagePublisher publisher)
    {
        _publisher = publisher;
    }

    [HttpPost]
    public async Task Post([FromBody] OrderInfo orderInfo)
    {
        // Add internal metadata to the OrderInfo object 
        // we received, or any other business logic
        orderInfo.OrderTime = DateTime.UtcNow;
        orderInfo.OrderStatus = OrderStatus.Recieved;

        // The updated OrderInfo object will also be serialized as the SQS message
        await _publisher.PublishAsync(orderInfo);
    }
}

Sample handler:

// See later in the design for how this was configured and mapped to the queue
public class OrderInfoHandler : IMessageHandler<OrderInfo>
{
    public async Task<MessageStatus> HandleAsync(MessageEnvelope<OrderReceived> message, CancellationToken cancellationToken = default(CancellationToken))
    {
        // Here we're reading from the message within the metadata envelope
        var productId = message.Message.ProductId;
        
        // Here we can do our business logic based on what is in the message
        await UpdateInventory(productId);
        await PrintShippingLabel(productId, message.Message.CustomerId);

        // Indicate that OrderInfo has been processed successfully
        return MessageStatus.Success;
    }
}

On either this issue or on specific section(s) of the design in the PR 41, please comment with:

• What are your thoughts and feedback on the proposed framework design?
• What are your thoughts on the initial MVP scope?
• Which AWS messaging services do you use?
  • Amazon SQS
  • Amazon SNS
  • Amazon EventBridge
  • Amazon Kinesis
  • Amazon MQ
  • Other (specify which ones)

[ashovlin]

We've just pushed a revision to the design document with:

• Expanded the community library section at the top to clarify the niche we see this framework filling.
• Multiple commenters suggested CloudEvents, which admittedly wasn't on our radar initially. The JSON Event Format for CloudEvents is very similar to the envelope schema we had originally proposed, so we're now exploring using the CloudEvents format.

[ashishdhingra]

Related issue aws/dotnet#42 which has discussion thread.

[normj]

Closing this hear since the discussion is happening on aws/dotnet#42 and we now have our own separate repository for this work at https://github.com/awslabs/aws-dotnet-messaging.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.