feat: Introduce a proof-of-concept field Writer for writing to "native field" as well as "foreign field" transcripts

[huitseeker]

The problem

We wanted a proof-of-concept of a set of traits that allow, for each concrete type:

• modelizing how to absorb it to a transcript able to ingest field elements of the native field,
• also modelizing how to absorb it to a transcript able to ingest field elements not in the native field, through a limbing conversion,
• not having to re-explain the conversion between sorts of field elements for every struct: we ideally want to explain field ingestion once and be done with it.

We're working with the following restrictions:

• the Rust type system does not allow trait specialization, or any form of ambiguous trait dispatch.

What's in this PR?

A PoC suite of traits explaining how structs can have a "serialization" to their "native" notion of field element, the transcripts can have a "native" notion of field element they accept, and how we can have a generic struct apply a systematic conversion between those two notions in order to define a general way to absorb the structs into the transcripts.

When the "native" fields on each end (struct and transcript) match, the conversion resolves to something trivial. That conversion is then made accessible to any struct that has defined its "native serialization" by way of an extension trait.

In Detail

• Created src/transcript.rs with concepts and traits for field operations.
• Defined a new Limbable trait for encoding translation between fields.
• Implemented FieldWriter and FieldWritable traits for respectively writing and producing field elements.
• Presented FieldWritableExt extension trait to accommodate different target field types.
• Added FieldEncodingWriter structure, a translating field writer.
• Integrated smallvec library with const_generics feature to implement it efficiently.
• Added tests for the src/transcript.rs module and its functionalities.

[adr1anh]

Amazing work! Can't wait to use this throughout.

How easily can we adapt this to write to a u8 buffer, so that we could use the same API for the pp-SNARK's keccack transcript?

[huitseeker]

@adr1anh does the last commit de9f873 answer your question precisely?

[adr1anh]

Thanks @huitseeker ! It's nice how easy you made it to serialize to a bytes buffer by reusing the io::write trait.

I’m realizing though that this may lead to a slightly less optimal verifier, if the latter is not restricted to only field elements (e.g. in Solidity).

With our current stack, the Solidity verifier has to run two interactive protocols to verify an NIVC proof.

• The folding verifier which uses the field-only Poseidon transcript, to correspond with the implementation in the circuit.
• The CompressedSNARK, where we would want to either create a proof using Poseidon to facilitate verification inside a circuit, or a proof using Keccack (or another pre-compile enabled hash function).

For the latter, we'll want to modify the "serialization" strategy to match the verification environment.

• Both Scalar and Base fields are "equivalent" in the sense that there is no difference in computation we perform on them. Both should be serialized using as_repr instead of limbing
• Commitments on the primary curve will not be pre-hashed and can be absorbed as-is. Moreover, we might want to write the "compressed" version instead.

What I was thinking originally is that we could get away with having a different implementation of write depending on whether the Writer writes Field or u8 elements.

I hope this extra context helps, but I’m also fine with focusing on the Poseidon use-case for now if this is becoming too complex!

[huitseeker]

@adr1anh is 3f516aa working for you?

[huitseeker]

This seems less useful to @adr1anh this days. Closing for clarity, @adr1anh feel free to re-open.