What is the best place to store an annotation set in an EPUB file?

[llemeurfr]

The current proposal is https://github.com/readium/annotations?tab=readme-ov-file#31-in-epub.
It stipulates that annotations.json is stored in META-INF.

But Open Annotation in EPUB stipulates that "the Annotation Document and all local resources are listed in the Package Document manifest and included in the EPUB Container like any other Publication Resources."

[llemeurfr]

META-INF currently contains a closed set of "configuration" files and extending it with a new file may raise interop issues (even if the EPUB spec does not state anything about the extensibility of META-INF).

Should we instead reference the annotations in the manifest and consider annotations.json as a classical EPUB resource, like media overlays? The big disadvantage is that it would mean modifying the OPF (which is handled by the publisher) and invalidating any EPUB signature.

Or should we instead place the file in another well-known directory in the zip?

Should we impose an XML translation of the JSON file to be more in line with other EPUB plumbing?

[mickael-menu]

We already have prior art with /META-INF/license.lcpl but also /manifest.json for EPUB files augmented with an RWPM.

I think META-INF makes sense, as the annotations document is not part of the publication, and it shouldn't interfere with EPUBCheck. Also the spec says:

All OCF Abstract Containers must include a directory called META-INF. This directory contains the files specified below. Files other than the ones defined below may be included in the META-INF directory; OCF Processors must not fail when encountering such files.
https://idpf.org/epub/30/spec/epub30-ocf.html#sec-container-metainf

Not that EPUBCheck matters post-production, IMHO.

Should we impose an XML translation of the JSON file to be more in line with other EPUB plumbing?

I would not use two different serialization formats for the same model. And as Readium Annotations will be used outside EPUB, and even Open Annotations in EPUB uses JSON, I think it's safe to use only JSON.

[HadrienGardeur]

We already have prior art with /META-INF/license.lcpl but also /manifest.json for EPUB files augmented with an RWPM.

That's a good point.

They're quite different from one another though:

• /META-INF/license.lcpl is mainly injected by reading systems into the package after acquiring the publication
• whereas /manifest.json is meant to be added into the package during production and should be referenced in the OPF as well

For the sake of consistency, I think that annotations.json should therefore end up in META-INF as well.

Since we're also talking about the ability to support multiple annotation sets (see #6), we'll also need to take into account whether we can truly handle our use cases with a single file.

[llemeurfr]

In the current model, the Annotation Set is just a vehicule for storing annotations in a file. The "shape" of an Annotation Set disappears on import; it is replaced by a tag (this was in fact the first reason why we introduced a tag in the model).
Therefore annotations.json contains EVERY annotation the user as decided to export (Note that he can filter such export by tag, see https://github.com/readium/annotations?tab=readme-ov-file#44-exporting-annotations-in-a-publication).

[HadrienGardeur]

I think that this is too focused on a file-based approach.

IMO, the concept of a set is orthogonal to the concept of tags. I'm fine avoiding the trap of attribution/authorship by relying on sets (which have their own creator property), but I don't think that we can mash everything together into tags without creating a giant mess.

[llemeurfr]

I insist on avoiding considering an Annotation Set for more than what it is: a temporary envelope for individual annotations. An Annotation Set has a generator but no creator.

[mickael-menu]

But Open Annotation in EPUB stipulates that "the Annotation Document and all local resources are listed in the Package Document manifest and included in the EPUB Container like any other Publication Resources."

I'm not sure I understand why they consider that an annotations document is a "Publication Resource"? Is their use case for stuff like author's notes?

[HadrienGardeur]

If an author or an editor includes such annotations in the package, it would make sense to reference them in the OPF.

This could be a way of identifying authored annotations vs user annotations as well.

[llemeurfr]

It would be an added complexity and - franckly - I don't see the publishing industry going there is the next decade.