Extracting request body as bytes

[danielkza]

Hello,

I'm trying out xpresso for a project in which I need to handle Github webhooks, which means I need to be able to take a request payload as bytes to generate a signature to validate against the one provided in the request headers.

I couldn't find any way to directly extract the payload as bytes, while also maintaining the possibility of extracting it as Json with FromJson for further use.

Should I be using a middleware or something different to do the validation?

Thanks in advance!

[danielkza]

Oh, it seems Files are the answer to this. It was a bit hard to guess this was the case, as in my conception what I'm doing is unrelated to files, but I guess it makes sense since I need the raw bytes of the payload. I'll try that and report how it goes!

[adriangb]

Do you think we should rename File to RawBody or something like that?

[danielkza]

That could be a good solution, I tried searching the docs for raw, payload and a few other keywords but I completely glossed over the Files section because in my mind, I was not attempting to save anything to a file at all.

Maybe just keep the existing names but add a RawBody alias?

Thank you for responding so quickly!

[adriangb]

There's a couple different ways to extract a body as bytes:

• FromFile[UploadFile]: This will extract the entire request into an UploadFile and consume the request body. Note that unlike FastAPI this will not be a multipart request (that's handled using FromMultipart[SomeFormModel]).
• FromFile[bytes]: This will extract directly into bytes but is otherwise the same as above.
• FromFile[AsyncIterable[bytes]: this will give you a handle to the body stream (same as Request.stream())
• Annotated[bytes, File(consume=False)]: this is probably what you want. It will extract the request body to bytes without consuming it. You could then use that with Annotated[SomePydanticModel, Json(consume=False)] to also get it as JSON (note that you'll have to mark both as consume=False, it might be a nice little QOL improvement to evaluate consume=False bodies before consume=True but that's not currently in place).
• Annotated[UploadFile, File(consume=False)]

Where the imports for the are:

from typing import Annotated  # or from typing_extensions import Annotated for py<3.9
from xpresso import UploadFile, File, FromFile, FromJson, Json, UploadFile, FromMultipart

[danielkza]

Thank you Adrian, I had found Annotated[bytes, File(consume=False)] and it worked as I expected. I had not added consume=False to the JSON loading, and it worked, but maybe accidentally if it depends on the order the parsing is done.

[adriangb]

Yeah it would depend on the accidental order of parsing, kinda an implement detail as of now that I wouldn’t rely on.

[danielkza]

Using File I can nicely compartmentalise extracting all the required data to validate a webhook, very nice end result:

class WebhookInput(BaseModel):
    delivery: Annotated[str,
        HeaderParam(alias="X-GitHub-Delivery")]
    event: Annotated[str,
        HeaderParam(alias="X-Github-Event")]
    payload_bytes: Annotated[bytes,
        File(consume=False)]
    payload: Annotated[Mapping[str, Any],
        Json(consume=False)]
    signature: Annotated[Optional[str],
        HeaderParam(alias="X-Hub-Signature-256")]
        
async def handle_webhook(input: WebhookInput) -> Any:
    ...

[adriangb]

Awesome!