Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clarify transcoding capabilities and requirements #311

Open
toddjonker opened this issue Apr 18, 2024 · 1 comment
Open

Clarify transcoding capabilities and requirements #311

toddjonker opened this issue Apr 18, 2024 · 1 comment
Labels
documentation specification 1.1 Ion 1.1 specification work

Comments

@toddjonker
Copy link
Contributor

One fundamental capability of Ion 1.0 is that any document can be transcoded between text and binary without access to any shared symbol tables, even without evaluating local symbol table directives. For the moment, I'll call this capability "standalone transcoding" since the document is all that's needed to support transcoding.

Note that this encompasses the basic case of pretty-printing a binary document for human consumption; any Ion 1.0 binary document can be pretty-printed as text in a standalone fashion. It might be full of SIDs, but it's structurally accurate.

It's recently become clear that 1.1 has broken this capability:

  • E-expressions cannot be parsed from or written to binary without access to the module that defines them. If imported from a shared module, that module must be accessible before a single byte of the E-expression's body can be produced or consumed.
    • The above applies recursively through macro-shaped parameters.
  • Any consumption of a document requires top-level E-expressions to be expanded at least to the point of determining whether any output values are directives.
  • Wherever a macro_table (either in a directive or a module) exports a full module by-name, eg (macro_table SomeModule), that module must be available in order to determine the number of addresses to allocate.

Since the binary pretty-printing has long been a significant part of Ion's elevator pitch, I think we need to have a clear story here. At the very least, we should work through the encoding conditions required to enable things like standalone transcoding and pretty-printing, so we can advise users what to do or avoid if they want their documents to retain that property.

@tgregg (I think) suggested that we add a distinct directive syntax that guarantees the document (or at least the segment) supports desirable transcoding properties without access to shared modules, and I think that's a very good idea.

@toddjonker
Copy link
Contributor Author

Relevant discussion and spec passage at #308 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation specification 1.1 Ion 1.1 specification work
Projects
None yet
Development

No branches or pull requests

1 participant